System and method for providing three-dimensional graphical user interface

ABSTRACT

Methods and systems are provided for providing an improved three-dimensional graphical user interface. In one embodiment, the method generally comprises: receiving an input from an end user, and capturing computing output from at least one computer source in response to the received end-user input. The computing output can be presented as two or more objects within a three-dimensional virtual space displayed to the end user. In one embodiment, the method further comprises generating a timeline that includes an icon for each object presented within the virtual space. In another embodiment, the method further comprises providing a database for storing and categorizing data regarding each object presented within the virtual space.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.11/531,676, filed Sep. 13, 2006, now U.S. Pat. No. 7,735,018, whichclaims priority pursuant to 35 U.S.C. §119(e) to U.S. ProvisionalApplication No. 60/717,019, filed Sep. 13, 2005, which application isspecifically incorporated herein, in its entirety, by reference.

COPYRIGHT NOTICE

This patent document contains material that is subject to copyrightprotection. The copyright owner has no objection to the reproduction ofthis patent document or related materials as they appear in the files ofthe patent offices of the United States or other countries, butotherwise reserves all copyright rights whatsoever.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention is directed toward graphical user interfaces foroperating and accessing information on a computer, and moreparticularly, to a three-dimensional (“3D”) interactive computinginterface and sorting interface comprising information from real-timeand static sources, including, but not limited to, meta search resultsfrom the Web; information from APIs, webservices, search engines,application programs, and networks; and files on the end user's desktop.

2. Description of Related Art

Currently, people use computers by inputting information into thecomputer to achieve a given output. Often this can be a series oftedious steps (mouse clicks and keyboard inputs) to run applications anddocuments or navigate to information. To get to new computingexperiences, people often have to close their current applications anddocuments, hide them or overlap them on a finite desktop by drawing themon top of each other, and then mine through folders within folders tofind them again at a later date. The user's desktop is finite, and onemust redo the same tasks over and over again. This wastes time by (i)requiring many mouse clicks to open and close documents, (ii) requiringone to remember all the combinations of programs and documents one mightneed for a given purpose and (iii) requiring one to create elaboratehierarchical folder systems to aid in the process of storing andrecalling applications and documents. This is primarily due to thelimited space the end user has on their desktop.

People currently compute within operating systems that present computeroutput, such as documents, applications, and operating system'sinterface in a 2D (two-dimensional) visual display. After initiallybeing loaded into the computer by the boot program, the operating systemcontrols all the other programs in a computer. Typically, the componentof the operating system that summons the style in which this output isdisplayed is called the GUI or graphical user interface. A successfulGUI will use screen presentations including metaphors that utilizegraphic elements such as icons to make an operating system's input andoutput easier to manage. Most computer operating systems incorporate aGUI that utilizes two-dimensional graphics to capture, process, andoutput all input from an end user in a 2D form—having height and widthonly.

This output is usually confined within a window that is drawn on afinite-sized desktop, i.e., the working area of a computer, that has agiven length and width. When the computer's output exceeds this finiteworking graphical area, elements of the GUI (the windows) are typicallydrawn on top of each other such that the GUI components overlap oneanother other. In some operating systems, a shadow is drawn beneaththese overlapping windows on the desktop to make them appear as if theyhave depth. This technique allows an end user to identify theoverlapping windows more easily.

We live in a 3D (three-dimensional) world where we see that objects notonly have a horizontal position (x) and vertical position (y) but alsohave depth (z) that is also known as time, according to thethree-dimensional coordinate system of mathematics. This notion ofexpressing depth or time in a visual computer metaphor is important forthe creation of a visual history of the end user's computing sessions.By plotting new output of the computer (instead of replacing) in avirtual space that does not overlap or substitute what exists on thefinite desktop, a new virtual space through depth and time is created.For example, if one were to pull up the webpage for the URLhttp://www.yahoo.com, and then click on a hyperlink (e.g., finance), thecurrent webpage in its window would be replaced by the webpage forYahoo! finance.

3D has shown itself in computing primarily in the following areas: (1)games, (2) CAD/medical visualization, and (3) virtual worlds. A virtualworld is a computer-simulated environment that its users can inhabit andinteract with via avatars. This habitation usually is represented in theform of two- or three-dimensional graphical representations of humanoids(or other graphical or text-based avatars).

The navigation window of many desktop operating systems use controls andbuttons to allow end users to navigate to other folders and windows inthe hierarchical structure of the file system. Often, in navigating tonew windows, the new windows replace the display of the current window.Accordingly, it would be very desirable to provide an improved graphicaluser interface that allows the user to efficiently navigate though avirtual space wherein groups of windows can be easily organized, stored,and retrieved.

SUMMARY OF THE INVENTION

The present invention addresses the shortcomings of the prior-artsystems and methods. In particular, the present invention is directed toa system and method for providing an improved 3D graphical userinterface.

In accordance with one aspect of the embodiments described herein, thereis provided a graphical user interface that uses the two-dimensional(“2D”) display of a user's computer to display three-dimensional (“3D”)objects in a simulated real-time 3D immersive Cartesian space.

In one embodiment, there is provided a system whereby new computingoutput occupies new virtual space near the original output, withoutlosing the original output. When an end user clicks on a hyperlink onthe webpage, there appears in the virtual space a new webpage that islinked to but does not replace the current webpage in its window;rather, the new webpage is drawn in a new virtual space. This way, theend user can visit past visual computing moments in time.

In accordance with another aspect of the embodiments described herein,there is provided a method for providing a three-dimensional graphicaluser interface, comprising receiving an input from an end user,capturing computing output from at least one computer source in responseto the received end-user input, and presenting the computing output asat least two objects within a three-dimensional virtual space displayedto the end user.

In one embodiment, the method further comprises generating a timelinethat includes an icon for each object presented within the virtualspace, wherein the icons are organized in linear chronological orderaccording to when the objects were presented within the virtual spaceand displaying the timeline within the virtual space. In anotherembodiment, the method further comprises providing a database module forstoring and categorizing data regarding each object presented within thevirtual space, providing a hyperlink within the database module torespective viewpoint of each object presented within the virtual space,and displaying the data regarding one or more of the objects within thedatabase module presented along with virtual space.

In accordance with another aspect of the embodiments described herein,there is provided a system for providing a three-dimensional graphicaluser interface, comprising a display screen, an input device forreceiving an input from an end user, a processor module operativelycoupled to the display screen and the user input device, and a memorymodule operatively coupled to the processor module. The memory modulepreferably comprises executable code for the processor to capturecomputing output from at least one computer source in response to thereceived end-user input and present the computing output as at least twoobjects within a three-dimensional virtual space displayed on thedisplay screen.

In one embodiment, the memory module further comprises executable codefor the processor to generate a timeline that includes an icon for eachobject presented within the virtual space, wherein the icons areorganized in linear chronological order according to when the objectswere presented within the Cartesian space, and display the timelinewithin the virtual space. In another embodiment, the memory modulefurther comprises executable code for the processor to provide adatabase module for storing and categorizing data regarding each objectpresented within the virtual space, provide a hyperlink within thedatabase module to respective viewpoint of each object presented withinthe virtual space, and display the data regarding one or more of theobjects within the database module presented along with virtual space.

In accordance with another aspect of the embodiments described herein,there is provided a system for providing a three-dimensional graphicaluser interface in a computer network, comprising a server connected tothe computer network and a user-interface application executing inassociation with the server to provide the functions of receiving aninput from an end user, capturing computing output from at least onecomputer source in response to the received end-user input, andpresenting the computing output as at least two objects within athree-dimensional virtual space displayed to the end user.

In one embodiment, the user-interface application executing inassociation with the server further provides the functions of generatinga timeline that includes an icon for each object presented within thevirtual space, wherein the icons are organized in linear chronologicalorder according to when the objects were presented within the Cartesianspace, and displaying the timeline within the virtual space. In oneembodiment, the user-interface application executing in association withthe server further provides the functions of: providing a databasemodule for storing and categorizing data regarding each object presentedwithin the virtual space, providing a hyperlink within the databasemodule to respective viewpoint of each object presented within thevirtual space, and displaying the data regarding one or more of theobjects within the database module presented along with virtual space.

In accordance with another aspect of the embodiments described herein,there is provided a network system for providing a three-dimensionalgraphical user interface, comprising: a computer-server networkcomprising a plurality of servers in communication with each other; atleast one display screen operatively coupled to the computer-servernetwork; at least one input device for receiving an input from an enduser, the input device being operatively coupled to the computer servernetwork; and a software module for providing a series of screen displaysto the end user, the software module being accessible by one or more ofthe servers of the computer-server network. The software modulepreferably comprises instructions for directing the servers to capturecomputing output from at least one network source in response to thereceived end-user input and to present the computing output as at leasttwo objects within a simulated three-dimensional Cartesian spacedisplayed on the display screen.

In one embodiment, the software module further comprises instructionsfor directing the servers to generate a timeline that includes an iconfor each object presented within the Cartesian space, wherein the iconsare organized in linear chronological order according to when theobjects were presented within the Cartesian space and to display thetimeline within the Cartesian space. In another embodiment, the softwaremodule further comprises instructions for directing the servers toprovide a database module for storing and categorizing data regardingeach object presented within the virtual space, to provide a hyperlinkwithin the database module to respective viewpoint of each objectpresented within the virtual space, and to display the data regardingone or more of the objects within the database module presented alongwith virtual space.

In accordance with another aspect of the embodiments described herein,there is provided a computer-readable recording medium for storing acomputer program that makes a computer execute: receiving an input froman end user; capturing computing output from at least one computersource in response to the received end-user input; and presenting thecomputing output as at least two objects within a simulatedthree-dimensional Cartesian space displayed to the end user.

In one embodiment, the computer program makes the computer generate atimeline that includes an icon for each object presented within theCartesian space, wherein the icons are organized in linear chronologicalorder according to when the objects were presented within the Cartesianspace, and display the timeline within the Cartesian space. In anotherembodiment, the computer program makes the computer provide a databasemodule for storing and categorizing data regarding each object presentedwithin the virtual space, provide a hyperlink within the database moduleto respective viewpoint of each object presented within the virtualspace, and display the data regarding one or more of the objects withinthe database module presented along with virtual space.

In accordance with another aspect of the embodiments described herein,there is provided a 3D graphical user interface that takes a user fromone computing place to another while creating the illusion of infinitespace in three dimensions (“3D”). By capturing the output of the user'straditional two-dimensional desktop, the 3D GUI stages this outputseamlessly in a 3D space by plotting the windows or other graphicalrepresentations of programs in 3D. In one embodiment of the presentinvention, the 3D GUI anticipates what the user may seek next (forexample, the next webpage in a search result), eliminates dormantcomputing time, and puts the user in a reduced-click computingenvironment by automatically plotting the new computing experience whilevisually recording the old.

Because the 3D GUI creates the illusion of infinite space in 3D, it cancreate a visual history of the user's computing session, whereby theuser can visit past visual computing events (or a snapshot in time) bysimply navigating to previously recorded states or viewpoints.Accordingly, the 3D GUI can function as a visual chronological historyof the user's computing session, whereby the user can name the computerexperience they are currently having through their position (orviewpoint) in a 3D space and revisit it by recalling the name or titleat a later time. The 3D GUI automates computing by remembering where theuser left off last—visually—such that the next time the user requiresthe same series of inputs to achieve that same given output, the 3D GUIwill navigate the user through a 3D space that is the visual history ofwhere the user last left off.

In one embodiment, the 3D GUI can run as an Active X control within abrowser window on the desktop of a computer (in conjunction with aweb-browser program, such as Internet Explorer). In addition, thepresent invention can run as a stand-alone application or embeddedwithin an HTML page. For example, a 3D virtual space (that was saved inthe 3D GUI) of a series of photographs of a model wearing differentjewelry styles can be embedded on a jewelry e-commerce site by embeddingthe code for the Active X control version of the 3D GUI into the markuplanguage of the HTML page according to the syntax for the component.

The program may run, for example, in conjunction with Internet Explorer,other web browsers, or stand-alone applications. The 3D GUI allows theuser to create a 3D space on their computer or the web. Throughprogrammatic access or helper applications (e.g., represented byinteractive icons within the 3D space), the 3D GUI allows the user tolocate data, applications, files created by applications, desktopwindows, HTML pages, and 3D applications, and it facilitates or invitesthe graphical output of these files and programs in their interactive 3Dspaces. In general, the present invention displays graphics from theuser's 2D finite desktop in 3D infinite space while retaining thefunctionality of the 2D programs and documents. Users will be able touse these files and applications, without restrictions, within 3Dspaces.

In accordance with another aspect of the embodiments described herein,the 3D GUI allows the user to create one or multiple 3D spaces on thefly to facilitate the graphical output of files and applications. Forexample, the user may have the option of linking multiple 3D spacestogether, thereby creating a network of 3D spaces. Regardless of wherethe files or applications are located (e.g., within the same folder, ina subfolder, on a different computer, within the network, on a differentnetwork, across the Internet, etc.), the user will have full access tothe file through its native program, or to the website through thedefault browser. In this way, the 3D GUI allows the output of disparatecomputer programs to visually converge in one expandable, changeable 3Dspace.

A more complete understanding of the disclosed 3D graphical userinterface will be afforded to those skilled in the art, as well as arealization of additional advantages and objects thereof, by aconsideration of the following detailed description of the preferredembodiment. Reference will be made to the appended sheets of drawingswhich will first be described briefly.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A-1C provide a block diagram detailing one embodiment of theprocess for providing in improved three-dimensional graphical userinterface.

FIG. 2 is a flowchart illustrating the process for cyclically redrawinga 3D Cartesian space based on user input.

FIG. 3 is a flowchart illustrating an exemplary approach to processinginformation achieve interactive composite texture mapping, as well asinteractivity and persistency, in a virtual space.

FIGS. 4A and 4B provide a flowchart illustrating a process for creatinga 3D output of webpages or other content from hyperlinks.

FIG. 5A is a block diagram of one embodiment of a system for providing a3D GUI.

FIG. 5B is a block diagram of one embodiment of a system for providing a3D GUI in a computer network.

FIG. 5C is a block diagram of one embodiment of a network system forproviding a 3D GUI.

FIG. 6 is a block diagram detailing a process for sorting an array ofwebpages in a 3D stack.

FIG. 7 is a flowchart showing a process for sorting an array of webpagesin a 3D stack.

FIG. 8 is a flowchart for a process and system wherein multiple userscan simultaneously vie and modify virtual spaces in collaborativefashion.

FIG. 9 illustrates one embodiment of a 3D GUI application window.

FIG. 10 illustrates another embodiment of a 3D GUI application window.

FIG. 11 illustrates an embodiment of a 3D GUI application window with anopened database module.

FIG. 12 illustrates an arrangement of windows within the virtual spaceof one embodiment of a 3D GUI application window.

FIGS. 13A and 13B illustrate another embodiment of a 3D GUI applicationwindow with an opened database module.

FIG. 14 illustrates an embodiment of a 3D GUI having a paintbrushfeature.

FIG. 15 illustrates an exemplary arrangements of windows and timelineicons in a 3D GUI application window.

FIGS. 16A and 16B illustrate exemplary arrangements of 3D stacks andtimeline icons in a 3D GUI application window.

FIGS. 17A-17C illustrate exemplary arrangements of windows and timelineicons in a 3D GUI application window.

FIG. 18 illustrates another exemplary arrangement of 3D stacks andtimeline icons in a 3D GUI application window.

FIG. 19 illustrates an embodiment of a 3D GUI that deliversadvertisements into the virtual space for the end user.

FIG. 20 shows a 3D GUI with the Favorites helper application openedwithin the database module.

FIG. 21 shows a 3D GUI with the Searches helper application openedwithin the database module.

FIG. 22 shows a 3D GUI with an exemplary helper application for a musicfile opened within the virtual space.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

The present invention satisfies the need for a system and method ofproviding an improved three-dimensional graphical user interface. Inparticular, the present invention is directed to a system and method fordisplaying a three-dimensional graphical user interface by receiving aninput from an end user, capturing computing output from at least onecomputer source in response to the received end-user input, andpresenting the computing output as at least two objects within athree-dimensional virtual space displayed to the end user. The methodpreferably further comprises generating a timeline that includes an iconfor each object presented within the virtual space, wherein each of theicons are organized in linear chronological order according to when theobjects were presented within the virtual space. In the detaileddescription that follows, like element numerals are used to describelike elements illustrated in one or more of the figures.

Described herein is a system for (i) selectively capturing computingoutput and information (webpages, applications, documents, desktopsand/or anything that can be visualized on a computer) from disparatesources (local computer or network); (ii) allowing the captured outputand information to visually converge by staging or drawing it in acommon 3D virtual space; (iii) organizing this staged output in ameaningful way utilizing a novel 3D GUI to allow end users an easier andmore efficient way to organize, visualize, search, and sort thiscaptured output and information; and (iv) sharing these virtual spacesby saving them, publishing them to the web, e-mailing them, or allowingmultiple users to collaborate by simultaneously viewing and modifyingthem.

Described herein is a system for creating and managing this new 3Dcomputing experience, based on the existing infrastructure of a 2Doperating system's output. A preferred embodiment of this system,diagrammed in FIGS. 1A-1C, is described in further detail below. Inaccordance with one aspect of the embodiments described herein, there isprovided a system and method for creating a 3D interactive computinginterface and sorting interface that includes information from real-timeand static sources, including meta search results from the web;information from APIs, webservices, search engine outputs, applicationprogram outputs, and networks; and files on the end user'sdesktop/laptop in a unique interactive 3D interface.

“Meta search” refers to a search wherein a query is submitted to morethan one search engine or directory, and wherein results are reportedfrom all the engines, possibly after removing duplicates and sorting.“API” refers to an interface that enables one program to use facilitiesprovided by another, whether by calling that program or by being calledby it. At a higher level, an API is a set of functionality delivered bya programming system, and as such, the mix of APIs in a particularsystem explains what that system can do. “Web” refers a network ofservers linked together by a common protocol, allowing access tomillions of hypertext resources. It is also known as WWW, W3 and theWorld Wide Web. “Webservices” refers to a standard approach tointeroperating between different software applications, running on avariety of platforms and frameworks, over the Internet. It is softwarethat runs over a network and provides information services based on XMLstandards that are accessed through a URI (Universal ResourceIdentifier) address and an XML-defined information interface. “Realtime” refers to a transmission or data-processing mode in which the datais entered in an interactive session where an application can respondfast enough to affect later data input.

The invention provides a Graphical User Interface (GUI) that uses thetwo-dimensional display of an end user's computer to display information(e.g., webpages and other information mapped onto 3D objects) in asimulated real-time 3-D immersive Cartesian space. The program runswithin web browsers (e.g., Internet Explorer and Mozilla Firefox) or asa stand-alone application compatible with the local operating system.The 3D GUI program creates the appearance of a 3-D space within a 2-Dwindow on the desktop of a computer, as illustrated in the embodiment ofFIG. 10. The program can utilize a ubiquitous interactive and immersive3D rendering browser or player which will process 3D drawinginstructions based on higher-level language code (the program) writtenin the drawing language native to the browser.

The program creates what seems to be an infinite simulated 3-D Cartesianspace within the two-dimensional display or window of an end user'scomputer by abiding by the visual rule of perspective whereby geometryor objects that are supposed to be closer to oneself appear largerconcerning their spatial attribute and objects or geometry that arefurther away appear smaller, as shown in the exemplary embodiments ofFIGS. 10-12. The program simulates a 3-D space within a 2-D window byredrawing objects in the space relative to one another as determined bytheir perceived distance from the viewer. Objects that are supposed tobe further away are smaller whereas objects that are supposed to becloser are larger.

The program creates interactivity of the simulated real-time 3-Dimmersive Cartesian space. While the user is immersed in this 3D space,the program will take instructions from the user processed by the eventhandler presented by the Graphical User Interface initiated controlsthat can change their perspective or viewpoint (as defined as a locationor visual perspective in the local coordinate system orthree-dimensional space) by moving closer to it, away from it, changingtheir angle or both. Once the program receives user-initiated input tochange the visual perspective of the scene, the program will redraw thescene to reflect the user-initiated input, as well as changes to thevisual perspective.

In accordance with one aspect of the embodiments described herein, thereis provided a system and method for providing a three-dimensionalgraphical user interface. With reference to FIG. 5A, in one embodiment,the system 710 comprises a display screen 712 and input devices 714, 716for receiving an input from an end user. The system 710 furthercomprises a processor module 718 (operatively coupled to the displayscreen 712 and the user input devices 714, 716) and a memory module 720(operatively coupled to the processor module 718).

The memory module 720 preferably comprises executable code for theprocessor module 718 to capture computing output from at least onecomputer source in response to the received end-user input, and topresent the computing output as at least two objects within athree-dimensional virtual space displayed on the display screen 712. Inone embodiment, the memory module 720 preferably further comprisesexecutable code for the processor module 718 to generate a timeline 340that includes an icon for each object presented within the virtual space300, wherein the icons are organized in linear chronological orderaccording to when the objects were presented within the Cartesian space300. In another embodiment, the memory module 720 preferably furthercomprises executable code for the processor module 718 to provide acompass or database module 440 for storing and categorizing dataregarding each object presented within the virtual space, to provide ahyperlink within the database module to respective viewpoint of eachobject presented within the virtual space, and to display the dataregarding one or more of the objects within the database module 440presented along with virtual space 300.

In accordance with another aspect of the embodiments described herein,there is provided a system for providing a three-dimensional graphicaluser interface in a computer network. With reference to 5B, in oneembodiment, the system 730 comprises a server 732 connected to thecomputer network (734, 736, 738, 740) and a user-interface applicationexecuting in association with the server 732 to provide the functions ofreceiving an input from an end user; capturing computing output from atleast one computer source in response to the received end-user input;and presenting the computing output as at least two objects within athree-dimensional virtual space displayed to the end user. In oneembodiment, the user-interface application executing in association withthe server 732 preferably provides the functions of generating atimeline 340 that includes an icon for each object presented within thevirtual space 300 wherein the icons are organized in linearchronological order according to when the objects were presented withinthe Cartesian space 300 and displaying the timeline 340 within thevirtual space 300. In another embodiment, the user-interface applicationexecuting in association with the server 732 preferably provides thefunctions of providing a compass or database module 440 for storing andcategorizing data regarding each object presented within the virtualspace, providing a hyperlink within the database module to respectiveviewpoint of each object presented within the virtual space, anddisplaying the data regarding one or more of the objects within thedatabase module 440 presented along with virtual space 300.

In accordance with another aspect of the embodiments described herein,there is provided a network system for providing a three-dimensionalgraphical user interface. With reference to 5C, in one embodiment, thenetwork system 750 comprises: a computer-server network 751 comprising aplurality of servers (752, 754, 756, 758) in communication with eachother; at least one display screen 712 in operative communication withor operatively coupled to the computer-server network 751 (directly orindirectly); at least one input device (714, 716) for receiving an inputfrom an end user, the input devices (714, 716) being operatively coupledto the computer-server network 751; and a software module 760 forproviding a series of screen displays to the end user, the softwaremodule 760 being accessible by one or more of the servers (752, 754,756, 758) of the computer-server network 751.

The software module 760 preferably comprises instructions for directingthe servers (752, 754, 756, 758) to capture computing output from atleast one network source in response to the received end-user input, andpresent the computing output as at least two objects within a simulatedthree-dimensional Cartesian space 300 displayed on the display screen712. In one embodiment, the software module 760 further comprisesinstructions for directing one or more of the servers (752, 754, 756,758) to generate a timeline 340 that includes an icon for each objectpresented within the Cartesian space 300, wherein the icons areorganized in linear chronological order according to when the objectswere presented within the Cartesian space 300, and to display thetimeline 340 within the Cartesian space 300. In another embodiment, thesoftware module 760 further comprises instructions for directing one ormore of the servers (752, 754, 756, 758) to provide a compass ordatabase module 440 for storing and categorizing data regarding eachobject presented within the virtual space, to provide a hyperlink withinthe database module to respective viewpoint of each object presentedwithin the virtual space, and to display the data regarding one or moreof the objects within the database module 440 presented along withvirtual space 300.

Within the 3D immersive space that the 3D GUI creates, the user'sviewpoint can be changed, where “viewpoint” is defined as a specificlocation or perspective in the local coordinate system (3D space) fromwhich the user can view the scene or file. As such, an interface calleda compass 440 can be used to help the user name, map, and navigate theviewpoints in a 3D space, as illustrated in the exemplary embodiment ofFIG. 11. Here, the compass or database module 440, which is located tothe left of the display of the virtual space 300, can be used to recordthe user's current viewpoint at any time while immersed in the 3D space.For example, the entry 442 shows the viewpoint “Yahoo!” indexed in thecompass 440. The compass 440 can be used to assign one or multiple namesto the recorded viewpoint, and/or to store the names of viewpoint(s) asone name in a collection of names in a relational database. The namesconstitute a map of the 3D space as well as a method to navigate themap. For example, in one embodiment, there is provided a linear map,called a timeline 340, having a plurality of icons (502, 504, 506, 508).The icons 502, 504, 506, 508 in the timeline represent viewpointsindexed in the compass 440 and correspond to the windows 510, 512, 514,518, respectively. The compass can also serve as the user interface tothe relational database of stored names/viewpoints. The compass can havedrop-down menus, wherein each menu is a record (in the relationaldatabase) that stores the name(s) and/or fields of viewpoints assignedby the user or automatically generated by the application program. Thecompass can be expandable to facilitate an infinite array of menus orrecords, thereby creating a table or group of records.

In the embodiment shown in FIG. 11, the explorer pane of the 3D GUIwindow serves as the compass 440 as it pertains to the Windowsenvironment. In this pane, one can see tabs labeled Saved Spaces 450,Desktop 451, Favorites 452, Web Browsers 453, Searches 454, ImagesSearches 455, Movies Searches 456, Searches 457, Pictures 458,Sound/Music 459, 3D 460, and Memos 461. These tabs representprogrammatic access or helper applications, which are described infurther detail below.

As shown in the embodiment of FIG. 11, the tab called Web Browsers 453is selected, revealing the Web Browsers menu below it, and the name ofthe viewpoint of the webpage (shown in the main window or virtual space)whose URL is http://www.yahoo.com and whose viewpoint name as it relatesto the compass 440 is “http://www.yahoo.com—Yahoo!” 442. Morespecifically, listed in the menu of the compass 440 are the names offour viewpoints of the webpages (shown in the main window or virtualspace) whose URLs are http://www.yahoo.com, http://www.google.com,http://www.ebay.com, and http://www.msn.com and whose viewpoint names asthey relate to the compass (and are listed as such) are“http://www.yahoo.com—Yahoo!” 442, “http://www.google.com—Google” 443,“http://www.ebay.com—ebay” 444 and “http://www.msn.com—MSN” 445. In thisway, the end user can use the programmatic access or helper applications(450-461) to have their content staged in a 3D virtual space, can havethe viewpoint representation of their content automatically indexed andorganized in the compass 440, and can have a linear representation ofthe graphical events in the compass 440 indexed on the timeline 340 via3D icons (502, 504, 506, 508).

In a preferred embodiment of the present invention, the helperapplications are adapted to display the output of files and programswithin windows (e.g., within the 3D spaces). In an alternate embodimentof the present invention, the programmatic access or helper applicationsare adapted to display information via customized interfaces whosegraphical designs look like the real-world objects they seek torepresent. For example, with reference to the embodiment of FIG. 22, auser may run a helper application or programmatic access for a musicfile. After clicking on the icon for the helper application forsounds/music, the 3D GUI will prompt the user to locate the music fileon either their local computer, a networked computer, or the World WideWeb. The helper application will then draw the interface (680, 682, 684)for the music file, preferably in the form of a compact disc, in the 3Dspace (300). The user can then interact with the graphicalrepresentation of the music file (e.g., the graphical representation ofa CD) to run and play the music file shown in interface 690.

The naming of stored viewpoints can exist as a single name in one menuin the compass or as a combination of multiple names in multiple menus.In one embodiment, each viewpoint is associated with only one name ineach menu. This way, the user can select multiple names in multiplemenus to create meaningful combinations of names that dynamicallyretrieve stored viewpoints based on the relationship of names selected.The user can edit the menus of the compass, thereby adding or removingcategories or expanding the menus. All of this can happen in real time,as the user authors and interacts with his or her 3D scene(s). The usermay also be able to combine two or more disparate compass interfaces,thereby creating a larger virtual map and navigation system for one ormore 3D spaces. A named viewpoint in the compass can link to a viewpointin the current scene, a viewpoint in another 3D file on the localdesktop, or a viewpoint in another 3D file hosted on the World Wide Web.This way, the compass creates a means of navigation that createsabstraction between viewpoints in any given files (local desktop files,files on the web, etc.).

In one embodiment of the present invention, the 3D GUI is furtheradapted to generate custom 2D maps of stored viewpoints and their namesfor specific menu items in the compass. Specifically, the 3D GUI creates3D icons for each viewpoint named and created via the compass and plotsthem into the 3D space with the respective names assigned to them assignposts, allowing the user to map all of the viewpoints for a 3D spaceand to draw lines among the viewpoints to see relationships amongdisparate viewpoints.

With reference to the embodiment of FIG. 11, there is provided a linearmap 340 (drawn on the bottom margin of the virtual space 300), in aninfinite possibility of maps, whereby the 3D GUI is adapted to expressthe map of stored viewpoints as 3D icons with their names (should theend user mouse-over them) for specific menu items in the compass 440expressed as a timeline 340. Here, the timeline 340 is a map whichrepresents the linear progress of animation from the first viewpoint tothe last viewpoint, should the end user click each 3D icon in such asequence, originally created and indexed in the compass 440. In essence,each 3D icon (502, 504, 506, 508) is a hyperlink or graphic that jumpsto a new location or viewpoint (when clicked).

The present system utilizes a unique graphical user interface called atimeline that allows end users to (i) index and keep track of, (ii)navigate to and (iii) replay every visual event and important actionthat occurs within a virtual space as it is happening (on-the-fly) bydrawing icons for these visual events in linear chronological orderwithin the virtual space (see FIG. 18). As used herein, “visual event”refers to a change in appearance to the 3D virtual space typicallycaused by adding information and output captured from other sources(automatically through programmatic access or manually by the end user)and drawn within the 3D virtual space (see FIGS. 1 and 2—boxes 22, 24,26).

In one embodiment, each new action, initiated as a result of input fromthe end user or otherwise initiated by programmatic access, that resultsin a visual event, is automatically stored in the compass and thenplotted as an icon on a dynamic timeline linear map 340 (see FIGS. 11,12, and 13A). Those skilled in the art will understand that it would beimpossible to list every possible kind of computing event that wouldconstitute a visual event that would be drawn as an icon on the timeline340. In one embodiment (shown in FIGS. 11, 12, and 13A), the timelines340 are drawn horizontally on the bottom margin of the page in astraight line. In another embodiment (not shown), the timeline can bedrawn vertically on the left or right margin of the page.

Each item plotted in the timeline is an icon that represents the actionthe end user initiated to cause a visual event for which the icon on thetimeline was created. For example, with respect to a horizontaltimeline, items to the left of a given item on the timeline occurredbefore the given item, whereas items to the right of the given item onthe timeline occurred after the given item. With reference to theillustrative embodiment of FIG. 18, the visual event (610) for whichicon 612 was drawn on the timeline 340 occurred before the visual event(620) for which icon 622 was drawn on the timeline 340. Similarly, thevisual event (630) for which icon 632 was drawn on the timeline 340occurred after the visual event (620) for which icon 622 was drawn onthe timeline 340.

This way, end users can go back to past computing experiences (what wasdrawn at some viewpoint or x,y,z coordinate in the virtual space at thetime an icon was created and added to the timeline) by clicking any iconon the timeline (in a linear order, random order, etc.) which binds theend user to a viewpoint for which information regarding the visual eventis plotted within the virtual space; in this case in its own unique 3Dor visual stack in the current embodiment.

With reference to FIG. 13A, in one embodiment, there is provided ahelper application called Desktop 451 that allows the user to drag anddrop items shown in the menu 441 of the database module 440 into thevirtual space 300. A dynamic linear map (drawn on the bottom margin ofthe virtual space) in the form of the timeline 340 is provided toexpress the actions of the end user's input as icons on the timeline340. Should the user click any icon on the timeline 340, the user'sviewpoint will change to the first item in the 3D stack (i.e., stackvisualized in 3D) created for the action or visual event. Ultimately,the timeline 340 is a linear map for all the items or output the helperapplications (450-461) capture and draw within the 3D GUI's stage orvirtual space 300.

In another embodiment, illustrated in FIG. 16A, an end user selects thehelper application called Yahoo! Search (520), and types a search term(e.g., “Albert Einstein” or “Thomas Edison”) into the helperapplication's text input field (530). The helper application gathers theinformation for the search request through a webservice or othertechnique (FIGS. 4A and 4B) and plots the search results in a unique 3Dstack 560, which is represented in the timeline 340 by an icon 562. Theplotting of the search results in its own 3D stack by a helperapplication constitutes a visual event or change to the 3D virtual spacefor which an icon is drawn and added to the timeline.

An end user can select another helper application called Yahoo! ImageSearch (520) and enter the same search term in the search field (530) ofthe Yahoo! Image Search text input field. The Yahoo! Image Search helperapplication gathers the information for this search request byinterfacing with Yahoo!'s webservices and then plots the search resultsin its own unique 3D stack 570 translated to the right (+x in the 3DCartesian virtual space) of the previously plotted 3D stack or stackvisualized in 3D. In addition, the drawing of the new visual stackwithin the virtual space constitutes a visual event whereby another icon572 representing stack 570 is added to the timeline (to the right of thelast icon 562).

With reference to FIG. 16B, should an end user create a web-browser pagewithin the virtual space (FIGS. 1A-1C—box 60) such ashttp://www.google.com through the “address”: command-line interface 464,whereby an end user can input a URL and have it added to the virtualspace (FIGS. 1A-1C—box 60), the new webpage would be drawn in its own 3Dstack 580 translated to the right (+x in the 3D Cartesian virtual space)of the previously plotted 3D stacks 560, 570. This would add anothericon 582 to the timeline (to the right of the last icon 572). If the enduser navigated within the virtual space and changed their viewpoint andselected the “record viewpoint” feature of the 3D GUI system (FIGS.1A-1C—box 50), the would records the x, y, and z position (viewpoint)within the virtual space, adds this to the database module, such asunder a tab called favorites.

As shown in FIG. 20, when the user clicks on the “record viewpoint” icon650, the system records the x, y and z position (“viewpoint”) within thevirtual space, and adds this to the compass 440 as an item 652 under thetab called favorites 452. The system preferably allows the user to namethe viewpoint and draw a 3D icon 654 within the 3D virtual spaceindicating the position of the stored viewpoint within the virtual spaceas a signpost. The system preferably draws the 3D icon 656 for the newlycreated viewpoint on the timeline 340 translated to the right of thepreviously drawn icon on the timeline as this would be categorized as avisual event. In this way, each visual event added to the virtual spaceis archived and expressed on-the-fly on the timeline through its icon.

Clicking each of the icons on the timeline from left to right insuccession would result in successive changes to the end user'sviewpoint. In this way, visual events within the virtual space arearchived as they happen and expressed on the timeline in linear order.The user can travel back in time by simply clicking the icon on thetimeline that represents the “visual events” that were drawn in thevirtual space, for which the icon on the timeline was created. Clickingthe item on the timeline would change the end user's viewpoint to theposition in the virtual space where the visual events were originallydrawn; in this case, the first item in each 3D stack.

Within the 3D GUI system, the addition of items to the timeline isdynamic and happens when new visual events are created. Furthermore, thetimeline becomes an important component of the GUI system when a virtualspace is viewed by an end user other than its author (either throughe-mail or visiting a published URL of the saved space on a server). Uponviewing for the first time a virtual space that another end userauthored, an end user can click the icons on the timeline from left toright in linear fashion and experience in chronological order what theauthor of the virtual space experienced as they created it. In this way,the timeline captures or records the visual changes of a virtual spacein chronological order so others viewing it can re-experience thevirtual space by clicking the icons on the timeline in succession fromleft to right.

The 3D GUI preferably utilizes an application program or run-timeenvironment that is used to execute code programmed for it based on thissystem. For example, the program can utilize an interactive andimmersive 3D-rendering browser that processes 3D drawing instructionsbased on higher-level language code (the program) written in the drawinglanguage native to the browser program. There are numerous programminglanguages as well as run-time environments/3D-rendering browsers thatcan be used to achieve this. The run-time environment or browser can be(1) a stand alone application, (2) an Active X control or object withina web browser, and/or (3) an embedded object in webpages.

This system or 3D interactive computing interface will create what isknown as a virtual space on the computer desktop for which it runsthrough the browser program. A virtual space is simply a program(running within the run-time environment/3D-rendering browser)simulating a 3D space within a flat 2D display by redrawing objects inthe virtual space relative to one another as determined by theirperceived distance from the viewer, FIG. 2. Objects that are supposed tobe further away are drawn smaller whereas objects that are supposed tobe closer are drawn larger.

Furthermore, the subject matter of a simulated 3-D Cartesian space drawnwithin the two-dimensional display or Window of an end user's computeris preferably redrawn in a cyclical fashion (FIGS. 1 and 2—boxes 22, 24,26) to refresh the scene such that changes to the objects drawn musthappen quickly enough based on the responses of the end user such thatthe experience feels truly interactive.

The information that is responsible for a virtual space is not unlikeany other file where the information that composes the file can bestored, named, and amended at will (FIGS. 1A-1C—box 94). In addition,because a 3D virtual space is used (1) to express the graphical userinterface it utilizes for input and output and (2) as a stage tovisualize the information to be sorted and searched on the system, manyof the commands that are part of an operating system's file system canalso apply to saved virtual spaces. In addition, each saved virtualspace (FIGS. 1A-1C—box 96) can act as a container for all of the itemsthat were added to the virtual space through the helper applications(FIGS. 1A-1C—boxes 32, 34, 36, 38, 40) that act as a bridge allowinginformation to stream into the system's virtual spaces. As one of themore popular functions of the Internet is to download and transfer filesfrom one computer to another, the notion of utilizing a virtual space tooutput and add files, content, and information into as a medium toe-mail or transfer these files is novel and useful. Those skilled in theart will realize that once a virtual space is archived as a file, it caneasily be e-mailed or sent via FTP to another server or computer aseasily as any other e-mail or file.

In the current computing paradigm, e-mail messages are sent to oneanother as messages with attachments of binary files such as pictures,videos, sounds, and executable files. Once an e-mail message isreceived, to view an attachment, the recipient must (i) select the file,(ii) decode each file using their e-mail software, (iii) download it,and (iv) launch the file in its own new window using a separate programthat created it or can view it. In contrast, in the 3D GUI, all thefiles, such as pictures, video, sounds, webpages, or other content, thatare added to a virtual space (see FIGS. 1A-1C—boxes 32, 42, 50, 58, 60,62, 72, 74, 84, 86, 88, 90, 92, 94, 104) can be e-mailed as a whole inone file. Once the recipient of the e-mail receives the e-mail, they canaccess the virtual space with all of its content with one click, byclicking the hyperlink to the saved space whether it is (i) attached tothe e-mail or (ii) saved on a server (FIGS. 1A-1C—box 98). Doing soopens the virtual space and stages all of the content without the enduser needing to open files separately one by one.

More specifically, once the virtual space is received via e-mail ordownloaded to an end user's computer from a server or via FTP on theWeb, the hyperlink to the saved space in the webpage responsible forshowing the e-mail or FTP file sees the embedded HTML <object> tag forthe Active X control or application that can execute the e-mailed ortransferred file with one click. In general, this tag containsinformation needed to execute the control, which can be located on theGUI server or elsewhere. In this way, the 3D GUI system (configured inthis embodiment of the invention as an Active X control) can run on anycomputer, allowing saved virtual 3D spaces with an end user's content tobe sent to any other computer via e-mail, file transfer protocol orother method and have all the content within the virtual spaceaccessible to an end user with one click.

In one embodiment of the present system, the 3D GUI program allows theend user to publish their spaces to a GUI server (FIG. 8—box 282), suchas by clicking a publish button or the like within the program. The 3DGUI program saves the file by uploading it or pushing it to the serverand creates an address for this file as a URL (FIG. 8—boxes 284 and286). One or multiple users (FIG. 8—boxes 290 ₁-290 _(n)) could visitthis published URL at the same time. Visiting this URL would launch the3D GUI Active X control on each of the end users' client computers andindependently download the most recent version of the file to each ofthe multiple users' computers (FIG. 8—box 288) so they could interactwith and use the virtual space. Initially, each end user would beexecuting the same version of the virtual-space file initiallydownloaded on their client computer. The server would keep an index ofall the end users who downloaded the file (known as the “multipleusers”) through network communication among the client(s) and the servercomputer.

Any change through a visual event to any of the multiple users' virtualspaces would be journalized in its own file. Here, visual event refersto a change in appearance to the 3D virtual space, typically caused byadding information and output captured from other sources and drawnwithin the 3D virtual space (FIG. 2—box 22, 24, 26). The term“journalizing” refers to a file system that logs changes to a journalbefore actually writing them to the main file. Each file for eachjournalized change for each new visual event from every client computerwould be pushed to the server (FIG. 8—box 296), through a networkconnection, and added to the original file as a journal entry. Once theserver received a journal entry to a file it would push the additions tothe original file to all the multiple users' client computers except theclient computer for which the change originated. In this way, the newvisual events or journalized files from all the multiple users would beupdated to the virtual spaces of all the multiple users. This processpreferably recurs quickly in a cyclical fashion, thereby allowing allthe changes from all of the multiple users to be reflected in real time.

The 3D GUI program provides interactivity of the simulated real-time 3-Dimmersive Cartesian space. While the end user is immersed in this 3Dspace, the program will take instructions from the user processed by theevent handler presented by the Graphical User Interface initiatedcontrols that can change their perspective or viewpoint, which refers toa location or visual perspective in the local coordinate system orthree-dimensional space. As such, the end user will be able to achievemultiple, unique viewpoints in a virtual space by moving closer to oraway from an object in the virtual space (e.g., a webpage), and/orchanging the angle or perspective of the webpage in the virtual space.

In one embodiment of the invention, the Graphical User Interface controlthat assists the end user in changing their perspective in the virtualspace is called a navigator, and it can be seen in FIGS. 11 and 12. Byclicking on the icons on the navigator, the end user can change theirperspective or viewpoint in the 3D virtual space. In the embodimentshown in FIGS. 11 and 12, (1) the “+” button on the navigator moves theend user forward along the z-axis in the 3D Cartesian space; (2) the “−”button on the navigator moves the end user backward along the z-axis inthe 3D Cartesian space; (3) the “up arrow” moves the end user up alongthe y-axis in the 3D Cartesian space; (4) the “down arrow” moves the enduser down along the y-axis in the 95

3D Cartesian space; (5) the “left arrow” moves the end user left alongthe x-axis in the 3D Cartesian space; and the (6) “right arrow” movesthe end user right along the x-axis in the 3D Cartesian space.

Once the program receives user-initiated input to change the visualperspective of the scene, the program will redraw the scene to reflectthe user-initiated input as well as changes to the visual perspective,as illustrated in FIG. 2. The program recalculates the shapes and sizesof objects or geometry in the scene/3D Cartesian space to reflect thelocation or visual perspective of the end user in the local coordinatesystem. In order to achieve a realistic real-time experience, theprogram will redraw the scene in a cyclical fashion.

The event handler is a part of a computer program created to tell theprogram how to act in response to a specific event (e.g., the clickingof a mouse, the dragging of a scrollbar, or the pressing of a button).The program's custom event-handling functions will be executed by theevent dispatcher, which is a part of the operating system that detectsgraphical user interface (GUI) events and calls functions in theexecuting program to handle those events (seeen.wikipedia.org/wiki/Event_handler).

The program recalculates the shapes and sizes of objects or geometry inthe scene/3D Cartesian space to reflect the location or visualperspective of the end user (based on the input gathered by the eventhandler) in the local three dimensional coordinate system. In order toachieve a realistic real-time experience, the program will redraw thescene in a cyclical fashion. In this way, the end user can control theirnavigation, position, and viewpoint in the 3D immersive space, givingthem the freedom to visualize any geometry within the 3D space at anyangle or viewpoint chosen (see FIG. 2).

In accordance with one aspect of the embodiments described herein, thereis provided a method and system for creating a 3D output of webpagesfrom hyperlinks via interactive meta search results from search-engineoutputs. With reference to the embodiment of FIGS. 4A and 4B, there isprovided a graphical user interface, such as an input interface orhelper application (box 162) within the 3D scene, that allows the enduser to input a search term or combination of search terms once an enduser is within their 3D space. The helper application or programmaticaccess can allow the end user to input not only the search term(s), butthe source from which the search results should originate (e.g., eBay,Yahoo!, Google, e-mail, desktop, MySpace, MSN, or any other availablesource of information (as shown in FIGS. 4A and 4B—box 164).Accordingly, the proper helper application must be customized to captureor interface the 3D GUI system with the origin of information. Mostsearch engines, portals and publishers of information on the world wideweb require different programmatic access techniques to expose theirinformation through webservices to allow other systems to connect withtheir information. Workers who are skilled in this art will understandthat slight changes must be made to bridge the 3D GUI to thisinformation without departing from the system. For example, the methodcan comprise connecting with http://www.google.com to facilitate thebridge of information into the 3D GUI system.

Similar to the way a search engine's crawler roams the world wide web byvisiting hyperlinks, storing the URLs, and indexing the keywords andtext of each page encountered, in one embodiment, the 3D GUI program canconduct a world wide web search on-the-fly, using these search terms inthe search system or website of the end user's choice by opening one 2DHTML page of the homepage of the chosen search engine (in this casehttp://www.google.com FIGS. 4A and 4B—box 184), drawn off screen(hidden) through the operating system's web browser control (FIGS. 4Aand 4B—box 184) as a child window within the 3D GUI. Our program willthen enter the search term into the off screen text input field of thesearch engine's homepage (FIGS. 4A and 4B—box 188), emulate a carriagereturn, retrieve the search results (FIGS. 4A and 4B—box 192), parse thesearch results found in each HTML page, identify each hyperlink that thesearch engine returned to the end user as a search result, and storethis in an array. In one approach, the program will open one new webpageor window in the 3D space (behind the search results page) for eachelement of this array or each hyperlink found on the search system'sresults page that would display the webpage found at the URL for thegiven hyperlink.

If the end user conducts a new search, the program will open a newbrowser window with the URL of the search system of their choice offscreen for each search phrase entered into the input interface, transferthe keyword phrase to this search page as a staging area and emulate thecarriage return. Once the search results appear, they can be broughtinto the program again. However, each new search result much have itsown browser window drawn off screen and then brought back into the 3DCartesian space.

In addition, it should be noted that the web browser control in theWindows operating system adds browsing, document viewing, and datadownloading capabilities to applications. Applications using thiscontrol will allow the user to browse sites on the World Wide Web, aswell as folders in the local file system and on a network. The webbrowser control is used in certain embodiments of the present invention.However, it will be noted that the 3D GUI system can utilize any part ofthe operating system shell or other component, other than the webbrowser control, as the method to capture and display output from thecomputer.

The 3D GUI program can open one new webpage or window in the 3D virtualspace (behind the search results page FIGS. 4A and 4B—box 202) for eachelement of this array or hyperlinks found on the search system's resultspage that would display the webpage found at the URL for the givenhyperlink. In this way, the 3D GUI is not unlike a search engine'scrawler except here the system visualizes the actual webpages,information and hyperlinks it encounters on-the-fly in a 3D virtualspace instead of storing it in a database as an index. In FIGS. 4A and4B—box 202, one can identify the search results page drawn in front oftwo webpages. The system as a default draws each new webpage in what wecall a “3D stack” (i.e., a stack visualized in 3D, sometimes referred toas a 3D stack) as shown where each new webpage occupies an x,y,zcoordinate similar to the position of the existing webpage; except it isdrawn further into the distance along the z axis (where it appearssmaller from the given perspective) and is translated on the x or y orboth x, y axis to allow the end user to see multiple webpages from anygiven perspective. For example, in the embodiment of FIG. 9, one can seea bird's eye view of four 3D stacks 302, 304, 306, 308 drawn in avirtual space 300 where each 3D stack (each containing about ten itemsin this embodiment) represents a new search.

With reference to the embodiment of FIG. 10, frames 366 and 371 providetwo search results conducted using a search engine (e.g., Google). Eachsearch results in the creation of its own 3D stack (360, 370), eachstack including about ten webpages. The dynamic creation of these 3Dstacks (360, 370) in the 3D virtual space 300 is the default gestaltthat the 3D GUI display engine uses to output the elements found withinthe 3D virtual space's organized output of information. In anotherembodiment, the display engine has the ability to load other customtemplates to achieve one or more different gestalts (e.g., 3D stacks,etc.).

With continued reference to FIG. 10, there is provided anotherembodiment of a navigator 380. By clicking on the icons on the navigator380, such as next page 382 and previous page 384, the end user caneasily change his/her perspective or viewpoint to coincide with themeaning of the buttons on the navigator 380. For example, clicking thenext page 382 on the navigator 380 binds the end user to a close-upviewpoint of the next page in a 3D stack (e.g., moving frompage/frame/window 361 to page/frame/window 364). Additionally, (a) the +button 390 moves the end user forward along the +z axis in the 3DCartesian space, (b) the − button 392 moves the end user backward alongthe −z axis in the 3D Cartesian space.

There are also provided a group of navigation buttons 396 near thebottom of the screen. Clicking the button 324 brings up the first pageof a 3D stack (e.g., page 361 of stack 360), whereas button 406 pulls upthe last page of a 3D stack (e.g., page 368 of stack 360). Statedanother way, clicking button 398 binds the end user to a close-upviewpoint of the first page in the 3D stack, whereas clicking button 406binds the user to a close-up viewpoint of the last page in the stack.Similarly, clicking button 400 brings up the previous page, whereasbutton 404 pulls up the next page in a given stack. Clicking button 402causes the program to take a snapshot of the current Cartesian spacearrangement of windows and objects, which makes it possible for the userto pull up the current Cartesian space at a later time. Clicking the“next 10 pages” button (now shown) draws the next ten pages in the 3Dstack, adding it to the original ten pages to create a total of twentypages in the 3D stack. Clicking the previous 10 pages button 408 bindsthe end user to a close-up viewpoint of the previous 10 pages in a 3Dstack (for example, moving from the twentieth page back to the tenthpage).

Also illustrated in the embodiment of FIG. 10 is a supplementalnavigation button 410 for changing his/her perspective or viewpoint inthe 3D virtual space 300. For example (a) the up arrow 412 moves the enduser up along the +y axis in the 3D Cartesian space, (b) the down arrow414 moves the end user down along the −y axis in the 3D Cartesian space,(c) the left arrow 418 moves the end user left along the −x axis in the3D Cartesian space and the (d) right arrow 416 moves the end user rightalong the +x axis in the 3D Cartesian space.

When an end user clicks the close button 420 on any texture map of a webbrowser control represented in the 3D stack, the 3D GUI removes the pagefrom the 3D stack, animates the next page after removed page forward toreplace position of removed page in stack, animates page afterpreviously animated page to original position of previously animatedpage before it was moved, animates forward next page in 3D visual stackto replace position of previously animated page, animates next page instack to position of previously animated page, and repeats this processuntil the end of the 3D stack or last element in the array is reached.

In addition, the 3D GUI can be easily customized in another embodimentof the invention to accommodate a formula to dynamically compute theposition or x,y,z coordinates of each webpage (or other item) drawnwithin a 3D stack to take on the overall look of what we call a customgestalt whereby the configuration or pattern (x,y,z coordinates in thevirtual space) of elements are so unified as a whole that it cannot bedescribed merely as a sum of its parts. For example, in one embodiment(not illustrated), a different gestalt or pattern for drawing webpagestakes on the shape of a three-dimensional cube where each webpage isplotted at the point of intersection of three sides of the cube(different than the column gestalt in FIG. 9).

In another embodiment, the 3D GUI retrieves images from a helperapplication which are then presented in a matrix wherein four picturesare drawn into two rows and two columns for each 3D stack along the xand y plane within the virtual space. In yet another embodiment, thevisual gestalt is of a matrix where sixteen pictures are drawn into fourrows and four columns for each 3D stack along the x and y plane withinthe virtual space's gestalt.

In accordance with another aspect of the embodiments described herein,the 3D GUI “gestalt” can be adapted to provide a “Links In/Links Out”feature in combination with a gestalt for any object or web page in thesearch results within the virtual space. First, should the user request,the program will automatically allow the end user to plot all of the webpages or objects that “link out” of any given web page or object withinthe 3D virtual space. Second, should the user request, the program willautomatically allow the end user to plot all of the web pages or objectsthat “link in” to any given web page or object within the 3D virtualspace.

The 3D GUI system's display engine can also be adapted to output acustom gestalt suitable for e-mail. The labels drawn in the 3D Cartesianspace, their positions and angles for Links In/Links Out can becustomized to display any titles which correspond to the kind of datapresented. Each coordinate or position in the 3D Cartesian space forplotting the information displayed in 3D stacks and Links In/Links Out,are variables that can be amended within a custom template that iscompatible with the program to alter the look of the gestalt in the 3DCartesian space. This can be accomplished by using xml encoded data inthe template that utilizes the extensible nature of the program. Thoseskilled in the art will be able to create these extensible data tablesthat work with the system. The program can handle multiple templates atonce through template files and switch between different themeson-the-fly at the request of the end user through their input from theinterface panel.

In one approach, each coordinate or position in the 3D Cartesian spacedisplayed in the 3D stacks are variables that are amended by a customtemplate that is compatible with the program to alter the look of thegestalt in the 3D Cartesian space. This is accomplished by using xmlencoded data in the template file that utilizes the extensible nature ofthe program. For example, a cluster of eight windows can be arranged atthe corners of a cube configuration, wherein the each window can berotated through the cube arrangement to sort through the pages in thecluster. The Links In and Links Out feature can be represented by linesegments that connect each cluster. It will be noted that a cube clusteris only exemplary and that clusters having variable numbers of pages todisplay are contemplated as well. For example, in another embodiment,five windows are represented in a cluster having a pyramidconfiguration, wherein the pyramid comprises a rectangular base withfour points and a single point located above the rectangular base.

In one embodiment of the invention, the 3D GUI can be a historicalvisual chronology of the user's computing session, whereby theapplication can archive the date and time of each new visual event byrecording the viewpoint at that date and time of the computing sessionand revisit these visual events by restoring the viewpoint at that dateand time. Here, we are calling a “visual event” as any change inappearance to the 3D virtual space usually caused by adding informationand output captured from other sources and drawn within the 3D virtualspace (FIG. 2—boxes 22, 24, 26). In one embodiment of the presentinvention a virtual space is created as shown in FIG. 21. Furthermore,the helper application called “Yahoo! Search” is utilized to conduct asearch and plot the webpages for the search result in the 3D virtualspace in its own 3D stack. The searches tab 454 in the compass 440 isselected showing the date and time that each search result or webpagewas plotted within the virtual space in the window pane below it inchronological order (see entries 660-672 in menu 441 of compass 440).

We are calling the “compass” an area in the program that can storeinformation in the 3D GUI that is related to the program such asviewpoints. The compass can have categories where information is stored.Changing the categories gives the end user access to differentinformation, allowing one to filter or discover information based on thecategory selected where the information resides.

Here we see that the events (674, 676) shown or drawn in the virtualspace 300 correspond to the archived date and time indexed for eachevent in the searches tab of the compass. Since 3D virtual spaces have a(i) horizontal position known as (x), (ii) vertical position known as(y) and (iii) a position of depth (z) which is also known as time, onecan see how it is possible to create a visual history of the end user'scomputing session by plotting new output in a new position further alongthe (z) axis and date and, time stamp it (e.g., entry 670 which readsAug. 8, 2006-6:00 p.m.) In fact, the system can be programmed to archiveany information combined with the proper operating system controls orother output at the next visual event to be drawn in the 3D virtualspace by the help of custom helper applications. For example, agraphical event could be the creation of a webpage in the GUI's 3Dvirtual space by typing a URL such as http://www.yahoo.com followed by acarriage return on the command line interface labeled “address:” 464.

The 3D GUI automates navigation in computing by remembering where theuser left off last—visually—such that the next time the user requiresthe same series of inputs to achieve that same given output, the 3D GUIwill navigate the user through a recorded 3D space that is the visualhistory of where the user last left off and the items that were outputinto the virtual space will be staged just as they were.

With reference to FIG. 13A, in one embodiment, the application the 3DGUI runs as an Active X control within the Internet Explorer webbrowser. The Saved Spaces tab can be selected in the explorer pane toreveal all the saved spaces in the menu below it. Clicking a saved spacewill load the 3D virtual space into the main window. Window 490 showsthe output of another helper application called Desktop 451 whose nameis shown as a tab in the compass 440. In the illustrated embodiment, asaved Microsoft Word document 500 is running in a window 490 within a 3Dvirtual space 300 alongside items 492, 494, 496, 498. All of the outputsof these items were captured through the input of the end user throughhelper applications using the method called “Interactivity andPersistency” (see exemplary approach shown in FIG. 3). This output savedhere as a virtual space 300 can be combined together from disparatesources and saved together as one in a virtual space 300.

Furthermore, the Microsoft Word document 500 whose output is running inthe 3D virtual space window 490 was input (into the 3D virtual space300) by the end user by drag-and-drop (FIGS. 1A-1C—box 78) from the menuof the helper application labeled Desktop 451, in the explorer pane ofthe 3D GUI application shown as a graphical event. Using the helperapplication called Desktop 451, any file, document, application ordesktop can be added to a 3D GUI virtual space by drag-and-drop (FIGS.1A-1C—box 78) in the same way the Microsoft Word document 500 was addedto the virtual space 300. Furthermore, once the file, document,application or desktop is added to the virtual space 300, it is fullyinteractive and functional and appears no different from, or close to,the original way the program functions when it was not in the 3-DCartesian space 300.

FIG. 13B shows a saved virtual space whose file name is called ncn (510)whose itemized output from helper application called “Yahoo! Search”(520) is indexed in the explorer pane 441 of the 3D GUI window. Clickingone of these indexed names (viewpoints) will bind the end user to aviewpoint created by the helper application that brings a favorableviewpoint/perspective of the output for this particular webpage in the3D virtual space to the end user's view. As such, each name indexed inthe explorer pane of the window (compass or database module 440) underthe searches tab 454 serves both as an index of the search resultsgathered by the helper application as well as a hyperlink or trigger toa favorable viewpoint within the 3D virtual space of each webpage withinthe search results.

In certain cases it may be difficult to interact with a file, document,application, desktop or other output within a 3D virtual space. This canbe the case if the end user is occupying an unfavorable viewpoint in thevirtual space where objects are drawn in skew within the virtual space.In such a case, the 3D GUI system uses a technique called “Bind to theHUD”, which involves bringing a file, document, webpage, application,desktop or other output into view by drawing it on the HUD. Thoseskilled in the art will recognize that the term “HUD” or “heads-updisplay” refers generally to a way of projecting information directlyinto a human's visual field. This technique was pioneered for militaryaviation, but has been used experimentally in other applications aswell. This can be accomplished by changing the viewpoint of the end userwithin the virtual space so it is directly in an end user's visualfield.

When the viewpoint of the end user within a virtual space has caused thewebpage to be drawn in skew, there will often be a distortion in shapeof the normal distribution toward one side or the other. In such a case,the 3D GUI system utilizes the Bind to the HUD feature whereby clickingan icon or bottom (analogous to the minimize in windows operating systemenvironment) triggers a change to the viewpoint of the end user withinthe virtual space so that the webpage is directly in an end user'svisual field, thereby making it easier to interact with. In oneembodiment, this is accomplished by revealing the 2D version of thewebpage that was initially hidden or drawn off screen and positioning itin a layer that is in front of the 3D virtual space such that the enduser can interact with this layer in 2D. Furthermore, the end user hasthe freedom to unbind to the hud or hide the 2D webpage again that wasinitially hidden or drawn off screen by clicking the appropriate button(again, analogous to the minimize button in the windows operating systemenvironment). As such, an end user can toggle or switch between 2D and3D for any selectively captured computing output and information(webpages, applications, documents, desktops or anything that can bevisualized on a computer) that was drawn within a 3D virtual space atwill by using this technique.

Since the 3D GUI takes advantage of seemingly unlimited space, theoutput of applications and documents need not be closed, hidden orfiled. They are staged and can permanently exist visually open (byrecording their output in a 3D virtual space) where they are and how theuser last left them. The 3D GUI does this by allowing the user to recordeverything they ever did, visually, and letting them revisit it throughunlimited space. Should the user require a new computing experience,they simply create more virtual space and plot new applications anddocuments within this newly created virtual space through helperapplications or programmatic access.

Should the user require an old computing experience, the 3D GUI changesthe graphical output of the screen to visually represent what they sawduring the old computing experience (e.g., data at a particularviewpoint recorded at a particular date and time or archived under afile name). This old computing experience can be saved in a file as whatwe call a “saved space”. A saved space is not unlike any other file asit stores the information that can be seen in a virtual space by the 3DGUI program at any time the end user chooses by utilizing the save spacecommand. In addition, the 3D GUI can systematically archive any newvisual change or addition to the virtual space by date and time andrecall what was seen in a past virtual space by date and time. This way,the 3D GUI lets one travel back (visually) in computing time.

In accordance with one embodiment of the present invention, the 3D GUIprovides full functionality and interactivity of the 2D display of auser's computer (including the selective and isolated capturing ofgraphical windows, desktops, HTML pages, and general program outputs)redrawn into a novel simulated real-time 3D immersive Cartesian space,whereby the 2D graphics are drawn or mapped onto 3D objects.

The 3D GUI invention is a novel system that offers a 3D stage to bridgeinformation into and handle this information from an operating system'soutput. The 3D GUI system disclosed here can be adapted to capture anyoutput from any operating system, in the language of the operatingsystem, regardless of any programmatic or structural changes to theoperating system, the way it outputs or the sequence of programmaticevents used to program the operating system or interact with it.

In one embodiment, the 3D GUI system implements interactive compositetexture mapping of operating system controls or other operating systemoutputs into infinitely immersive interactive 3D Cartesian space tofacilitate search, sort and a browsing GUI system of information (e.g.,but not limited to webpages, applications, documents, windows, networks,webservices, etc.).

Regardless of the (1) kind of information brought into (through theoutput of the operating system) the program whether it be webpages,pictures, windows, applications, desktop screens, pdf files, etc. or (2)the method by which the information is captured by the program whetherit be through APIs, Meta Search, or webservices, or (3) the programmaticaccess by which it is delivered (controls, windows, desktops, images,VNC) it is critical that the program bring the information into thesimulated 3-D interactive Cartesian space in such a manner that theinformation being brought in is fully interactive and functional andappears no different from, or close to, the original way the programfunctions when it was not in the 3-D Cartesian space.

In accordance with one aspect of the embodiments described herein, thereis provided a special control from the operating system such that theinformation being dealt with (e.g., webpages) functions properly in the3-D Space as it would in 3D. Accordingly, in the world of computing, a“CONTROL” is defined as an object that enables user interaction orinput, often to initiate action, display information or set values. Forexample, in order for a webpage on the World Wide Web to functionproperly in the operating system in a typical window on the 2-D desktopof an end user's computer, the language of that webpage (HTML) must beread by an operating system program or CONTROL in order for the webpagein question to be displayed and function properly. One such particularcontrol used by the Windows Operating Systems is called a Web Browsercontrol that deals with webpages as well as typical desktop windows.Currently, most Windows controls are visualized using a 2-D paradigm.The name of one such control is called MSHTML/webbrowser control forrendering HTML webpages and other content on the Windows desktop withina window.

Webpages, unlike pictures that the end user simply view, requireinteractivity to function properly in a virtual space. Because theviewpoint of an end user within a 3D interactive virtual space canchange, so to do the shapes and sizes of the objects being drawn changebased on the end user's navigation within the virtual space. If theobjects being drawn within a 3D virtual space having operating systemoutput such as controls mapped onto them, a special system must becreated to insure that the end user can interact with the mapped objectin a 3D virtual space with the same responsiveness of input and outputthat one would find in a 2D desktop. As such, described herein is ainteractive 3D composite texture mapping to serve as a visual bridge ofinformation from an operating system's 2-D output (e.g., theprogrammatic access for which the 2-D is being displayed utilizes acontrol, and all the information a control is capable of displaying) tothe 3-D interactive Cartesian space, wherein the represented objectsremains fully functional in the immersive space from any viewpoint thatthe end user selects (see FIG. 3).

As such, the program delivers full functionality and interactivity ofthe two-dimensional display of an end user's computer (including theselective and isolated capturing of graphical windows, desktops, HTMLpages, and general program outputs or any information a control candisplay or non-control item) redrawn into a novel simulated real-time3-D immersive Cartesian space whereby the two-dimensional graphics aredrawn or mapped onto three-dimensional objects.

While immersed in this 3D space, end users can see their two-dimensionalcomputer display components that were captured and redrawn and fullyinteract with them, creating an entirely new way of computing. Whileinteracting with their mapped or redrawn two-dimensional displays, endusers can change their perspective or viewpoint (as defined as alocation or visual perspective in the local coordinate system orthree-dimensional space) by moving closer to it, away from it, changingtheir angle or both.

In one exemplary embodiment, the visual output of an operating system'scontrol that one would normally find output on the 2D desktop (e.g.,webbrowser control/msHTML) is texture mapped onto 3D geometry (e.g., acube, pyramid, etc.) in the GUI system's 3D immersive virtual space.This is one operating system control chosen from an infinite library ofpossible operating system controls, APIs or any outputs whose output canbe captured by the 3D GUI and drawn onto any object in the 3D space. Inaddition, the origin and method for the visual output of an operatingsystem's control can be texture-mapped onto the 3D Geometry in the GUIsystem's 3D immersive virtual space regardless of what it is or how itwas captured.

With reference to FIG. 3, there is provided an Interactivity andPersistency Diagram illustrating the processing of information requiredto achieve both interactive composite texture mapping as well asinteractivity and persistency in a virtual space. In the diagram, thisprocess is initiated with the launching of the 3D GUI application whichcan be installed on either a client computer or a server computer.

The 3D GUI running on a server can deliver information and software toother computers linked by a network. Here, the client is the requestingapplication program or user in a client/server relationship. Forexample, the 3D GUI client application is effectively making clientrequests for information from the 3D GUI server over the Web. The 3D GUIprogram itself is a client in its relationship with the computer that isgetting and returning the requested information. The computer handlingthe request and sending back the information is the 3D GUI server.

In this embodiment, the 3D GUI application program will run locally onthe computer rather than as a web-application or network-applicationover a network connection. However, both configurations are possible.Should the invention execute on a server or other such computer, thiswould provide a user of the client computer to have access to othercomputers' operating system's output. Initially, the application islaunched as described by Launch Application in the diagram. A virtualspace is created, “Create Interactive 3D Cartesian Space”, on thecomputer desktop for which it runs through the program. Virtual space issimply a program (running within the run-time environment/3d renderingbrowser) simulating a 3D space within a flat 2D display by redrawingobjects in the virtual space. Depending on the application or computingpurpose at hand, the program will capture user input request based onthe intended program customization.

Through programmatic access, the application will transfer and initiateuser requests from the 3D virtual space to 2D desktop or directly to theoperating system of the computer through an API call. If this transferof user requests from 3D to 2D is done directly to the 2D desktopwithout an API call, the user requests are transferred bysimulating/emulating or reproducing the request or device input fromevent handler off-screen onto the hidden 2D mirror component andcapturing the response or change in output from the 2D mirror again in asynchronous fashion and mapping it back to arbitrary 3D geometry. We sayoff-screen because the 2D desktop or operating system control is hiddenfrom the end user to focus attention on the 3D virtual space. In thisway, end users can compute with the same outputs from their 2D desktopthrough a 3D virtual space that acts as 3D window into 2D output,setting the stage for computing with seamlessly unlimited virtual space.

Based on program customization, the application will “Determine properOS control to retrieve the kind of information requested” by theend-user to the purpose at hand. For example, this could include (butnot limited to): (1) opening a webpage, (2) opening a file or document,(3) launching an application, (4) create a window or (5) executing anyother program locally or over a network connection.

As such, once the proper operating system control is specified andlocated, an application program interface call is executed for thissystem control. In order to enable the usability and functionality ofthis control or other operating system application program interfacewithin a virtual space, the 3D GUI will blit or map visual output of OScontrol, bit map or API onto arbitrary 3D geometry. The phrase arbitrary3D geometry is used to clarify that this geometry can be unique to thetheme of the virtual space that the 3D GUI is customized to through atemplate that utilizes a specific gestalt.

It will be noted that the 3D GUI is not limited to one method forcapturing this map or “visual output of OS control” or simulating orpassing the user's request to the mirror as each operating system has aunique method for handling controls and their output.

Within the 3D virtual space, a device input event handler is utilized topass mouse clicks, cursor positions, keyboard input and movements fromthe operating system control mapped onto the 3D geometry to the mirrorcontrol running on 2D environment.

Currently, this method or cyclical process of capturing operating systemoutput and drawing it into a 3D virtual space is a workaround or manualprocedure implemented in order to overcome a shortcoming of theoperating system to the problem at hand. In another embodiment, theoperating system can incorporate this feature or similar to it with onesimple API call in the future at which time the 3D GUI system couldutilize this API.

This process of (1) mapping visual out of operating system control on 3Dgeometry in a virtual space, (2) scanning the device input event handlerand (3) passing this input to the mirror control running in 2Denvironment or directly to the operating system is repeated in acyclical fashion to create a real time experience of interacting withsaid operating system controls or operating system output in a virtualspace, regardless of the viewpoint or perspective of the end-user in thevirtual space. The 3D GUI application runs in real time in whichinformation is received and immediately responded to without any timedelay among the virtual space and the 2D map or operating system APIoutput. This synchronous communication is advantageous as too much delaywould make the system lag.

As one can discern from FIG. 3, in the situation that there is nooperating system control or programmatic access to capture the map orvisual output of an operating system control, a contingency isincorporated to periodically capture the on-screen output of anoperating system control or output (e.g., window) as a bit map from theframe buffer, video driver, operating system graphics API or videomemory by blit (to copy an image, or part of an image, from one place toanother). With further reference to FIG. 3, it will be noted that themethod and system for providing a 3D GUI as described herein allows thesystem to capture the visual output from any computer source regardlessof its origin or how it was output. For example, while boxes 128, 140,142 in FIG. 3 refer to OS Control and/or say “Execute Proper OS ControlSpecific to the Kind of information Requested . . . ,” it will be notedthat the invention is not limited to applications involving OS Control,but can relate more generally to applications involving ProgrammaticAccess or the like. For example, in certain embodiments, box 140 of theflowchart of FIG. 3 can read “Execute Proper Programmatic Access to theKind of Information Requested which will act as a Mirror to its 3DRepresentation.”

By capturing the output of the user's traditional two-dimensionaldesktop, the GUI stages this output in a seamlessly 3D space by plottingthe windows or other graphical representations of programs in 3D. In oneembodiment of the present invention, 3D GUI anticipates what the usermay seek next (for example, the next webpage in a search result),eliminates dormant computing time, and puts the user in a reduced-clickcomputing environment by automatically plotting the new computingexperience (in a new space, rather than overlapping it onto somethingelse) while visually recording the old. Because the 3D GUI creates theillusion of infinite space in 3D, it can create a visual history of theuser's computing session, whereby the user can visit past visualcomputing events (or a snapshot in time) by simply navigating topreviously recorded states. This can be accomplished because newinformation expressed graphically in a virtual space does not replaceold information by replacing it or overlapping it. Instead, it is drawnin new virtual space. Accordingly, the 3D GUI can serve a historicalvisual chronology of the user's computing session, whereby the user canname the computer experience they are currently having through theirposition(s) (or viewpoint) in a 3D space and revisit it by recalling thename or title at a later time.

While immersed in this 3D space, the user can see their 2D computerdisplay that was captured and redrawn and fully interact with it,creating an entirely new way of computing. While interacting with theirmapped or redrawn 2D display, the user can change their perspective orviewpoint (as defined as a location or visual perspective in the localcoordinate system or 3D space) by moving closer to it, away from it,changing their angle or both.

The 3D objects that are being drawn onto (or next to) can collectivelyrepresent a theme or backdrop for the content that is redrawn, such as ahouse, library or even neighborhood. For example, electronic music filesmay be mapped onto 3D pictures of CDs or Records, electronic videosfiles may be mapped onto televisions or movie screens, etc. (see FIG.22) The redrawing of 2D computer screens onto interactive 3D objectsincreases the visual computing space and allows the user to organize thecomputer output onto objects or near objects whose 3D visual constructmay represent the real world object the computer output is associatedwith.

In accordance with one aspect of the present invention, the 3D GUI isadapted to create a visual computing history, whereby normal changes toa 2D computer display output are drawn or mapped onto new 3D objects,rather than replace the current output or 2D display once a change ismade. For example, in one embodiment, when the user clicks on ahyperlink on webpage A, this results in the creation of a new webpage Binstead of replacing A. For example, as the user browses the internet byinteracting with a two-dimensional output mapped onto athree-dimensional object, pages that the user would normally hyperlinkto by clicking a link would replace the 2D computer display. The 3D GUI,however, is adapted to create an entirely new 3D object whose surface ismapped with the new display content that would normally replace theoriginal 2D interactive page.

By storing and archiving the date and time of each new graphical event,the user creates a visual chronology of their computing session, whichcan be recalled by clicking icons on a timeline where each icon on thetimeline represents the action of the end user at a specific date andtime. The programmatic access that is responsible for a past graphicalevent would also be archived. For example, in the case of viewing asecond HTML page that one hyperlinked to, the previous URL would besaved. This way, a user could always look back at what the computerpreviously displayed before it output the current view and interact andfunction with a previously saved computing state as representedgraphically. This process of capturing visually the computing history in3D can continue on indefinitely, whereby the 3D visual space can besaved, archived and revisited and the timeline would grow dynamicallywith additions of icons to represent this.

In accordance with one embodiment of the present invention, a compass(or database module) application is provided. The user is allowed toclick at least one button on the compass (while immersed in athree-dimensional interactive space) and assign one or multiple names toa viewpoint (as defined as a location or visual perspective in the localcoordinate system or three-dimensional space). The user may then viewthe three-dimensional image associated with the viewpoint and save thisviewpoint (or visual perspective) along with its corresponding name in afile whose content or data can be accessed by the compass. In oneembodiment of the present invention, the compass acts as an electroniccombination lock, whereby the combination or sequence of one or multiplenames assigned by the user and added to the dials of the compassidentify the stored location.

For example, consider the situation whereby a user is immersed in a 3Dinteractive space showing 3D objects of compact discs from The Beatles.To view the current viewpoint, the user could assign multiple names suchas MUSIC, ROCK, FOREIGN, BRITISH INVASION whereby each name would occupyone space on one of many dials of the graphical user interface. Byturning the dials to the names MUSIC, ROCK, FOREIGN, BRITISH INVASIONthe program would initiate a change of viewpoint to the 3D interactivespace showing the 3D objects of compact disks from The Beatles. The useof the interface feature for naming, storing and re-visiting viewpointsin a 3D space is universal and can be applied to a local file of a 3Dspace on the user's computer, a networked file or a file on the worldwide web.

In accordance with one aspect of the present invention, the user is ableto access stored viewpoints on a graphical interface called a compass,which may serve as an electronic combination lock. In this embodiment,the compass will open (or visually visit) a stored viewpoint when itsdials are turned through a predetermined sequence of positionsidentified on the dials' face by the appropriate names assigned to eachdial. Therefore, the compass can be used to connect a series ofviewpoints together and visually visit a series of connected storedviewpoints.

In accordance with one aspect of the present invention, the user isimmersed in a 3D space and provided with a button called new space orthe like. When this button is clicked, another file for a completely new3D interactive space is created, whereby the new 3D interactive space isaccessible from the current space by clicking a 3D object or portalbutton. The portal button, which electronically links the new space tothe current one, may be represented by a picture of a viewpoint orvisual perspective of the new space. By clicking the new space button,the 3D GUI will automatically create the portal button within thecurrent space that links to the new space. Furthermore, the 3D GUI mayalso allow the user to concurrently assign one or multiple names to thenew space, and add these name(s) to the database module (also known asthe compass).

Should a user create a new space and then save this space or e-mail it(FIGS. 1A-1C—boxes 96, 98) the 3D GUI may be adapted to automaticallycreate a table of contents page (FIGS. 1A-1C—box 100) by writing themarkup for an HTML page that describes the new space, the names assignedto the new space via the compass, and pictures of the new space throughsnapshots of different viewpoints or visual perspectives of the 3D file.All this information may be assembled into an HTML file that will beautomatically published or sent by file transfer protocol (FTP) to oneor many world wide web servers for promotion as to be found and crawledby the many search engines that traverse the world wide web.

One purpose of creating these HTML table of content pages for newlycreated 3D spaces on-the-fly and publishing them to the world wide webis so that they can be used as portal pages, whereby a user could (i)search the world wide web using keywords, (ii) find a table of contentspage whose subject corresponds to the keywords, and (iii) hyperlink theend user from a 2D table of contents page to a 3D interactive spacewhose subject matter corresponds to the table of contents page. Forexample, the HTML table of contents page may contain (i) the name of thenew space as the title of the HTML page, (ii) meta tags whose nameswould be taken from the names assigned to viewpoints of the compassinterface, (iii) a raster image or picture of the original viewpoint orvisual perspective of the new space, (iv) a list of all hyperlinks foundwithin the new space, (v) a description of the scene, (vi) the author ofthe scene, (vii) remote desktop connection settings and/or (viii) URLsto the desktops for all remote desktop connection links shown within thenew space.

Because the search engines that index information on the world wide webmainly do so for HTML (webpages), pictures and (in some occasions)files, the HTML table of content pages disclosed here serve as a doorwayfor saved 3D virtual spaces to be included in a search engine's index.Information about the 3D GUI Active X control that is responsible forrunning the control is coded into the HTML table of contents pagethrough use of the HTML <object> tag along with other informationdisclosed here that make up a table of contents page. In this way, the3D GUI system allows end user's to author content through their 3Dvirtual spaces, publish them on a server through a table of contentspage and insure that these pages can be crawled and seen by searchengines through the current paradigm of the world wide web where searchengines primarily use crawlers or spiders to examine hyperlinks andwords on webpages.

In accordance with one aspect of the present invention, the display of3D on a computer screen may involve a run-time execution model thatplays a previously programmed 3D space like a tape-recording in a windowor web browser as a one-way broadcast medium. The programming of this 3Dspace is accomplished by hand coding a program or using an editor whichprepares a file for the player. In one embodiment of the presentinvention, the helper applications can be accessed from the player(while an end user is using and immersed within a 3D space) through agraphical user interface of interactive icons that facilitate thedisplay output of a file or functionality of a program within windowswithin a 3D space.

In one embodiment, content output into the 3D GUI application's virtualspace is generated by running a helper application, such as eBay Searchor Yahoo Images Search. The resulting first product image output andtheir product information, generated by helper application whosefunctional diagram is shown in FIGS. 4A and 4B. The output preferablycomprises a linear map (e.g., drawn on the bottom margin of the virtualspace), whereby the 3D GUI is adapted to express the map of storedsearches as 3D icons with their names (should the end user mouse-overthem) for specific search items expressed as a timeline. Here, thetimeline is a map which represents the linear progress of animation fromthe first viewpoint of the first item in each 3D stack to the lastviewpoint of the first item in the last 3D stack should the end userclick each 3D icon in such a sequence, originally indexed in thecompass. In essence each 3D icon is a hyperlink; a graphic, whenclicked, that jumps to a new location or viewpoint in the currentvirtual space corresponding to the 3D icon. In this embodiment of theinvention, each new search results in a new 3D stack created for thatsearch result plotted in the 3D virtual space with its corresponding 3Dicon drawn.

For example, in the embodiment of FIG. 9, one can see three 3D icons(342, 344, 346, 348) in the timeline 340 representing four uniquesearches (i.e., rolex Daytona, ibm laptop, plasma tv, and treo 650) donewith the eBay Search helper application. Each unique search resulted inthe creation of its own 3D stacks 302, 304, 306, 308 (each stack showingten items at a time in this embodiment) as well as their own unique 3Dicons plotted in a timeline map 340 at the bottom margin of the 3Dvirtual space 300. In this embodiment of the invention, should the enduser click on any hyperlink or 3D icon, the 3D GUI would visually takethe end user to the viewpoint of the first eBay search result itemwithin its 3D stack.

In this way, end users can (1) successively enter in new search termsinto the eBay helper application, (2) press the carriage return toinitiate their searches, (3) visualize the search results plotted intotheir own unique visual column, (4) visualize the map of 3D iconsexpressed as a timeline on the bottom margin of the virtual space, (5)quickly navigate among (through the viewpoint of the first item in eachunique 3D stack) searches as they are expressed in 3D stacks by simplyclicking the hyperlink of 3D icons on the timeline and shuffle or sortthrough each item in each 3D stack by clicking the commands on thenavigator 320, such as next page 322, last page 324, next 10 pages 326,and last 10 pages 328. Also, by clicking on the icons on the navigator320, the end user can change their perspective or viewpoint in the 3Dvirtual space 300. For example, (1) the + sign 337 moves the end userforward along the +z axis in the 3D Cartesian space, (2) the − sign 338moves the end user backward along the −z axis in the 3D Cartesian space,(3) the up arrow 330 moves the end user up along the +y axis in the 3DCartesian space, (4) the down arrow 334 moves the end user down alongthe −y axis in the 3D Cartesian space, (5) the left arrow 336 moves theend user left along the −x axis in the 3D Cartesian space and the (6)right arrow 332 moves the end user right along the +x axis in the 3DCartesian space.

In the embodiment of FIG. 11, there is provided an address: command line464 interface or helper application whereby an end user can input a URLor address for a window, document or application in the local filesystem of a local operating system, etc. When an end user typeshttp://www.yahoo.com on the address field of the command line interface,the application then draws the HTML page through the proper web browsercontrol into the 3D virtual space as depicted. This process can berepeated indefinitely, entering in additional URLs on the command linelabeled address: to have them filed in a 3D stack. Four webpages (510,512, 514, 516) are created in a 3D GUI virtual space having sequentiallytyped in http://www.yahoo.com followed by a carriage return,http:www.google.com followed by a carriage return, http://www.ebay.comfollowed by a carriage return and then http://www.msn.com followed by acarriage return. The command line interface, helper application 464 isone way for the user to create and author HTML pages, desktop windows,documents, applications, vnc desktop, or anything else able to bevisualized in a virtual space on the fly at their behest.

In an alternate embodiment of the present invention, each helperapplication is adapted to display information via a customized 3Dinterface whose graphical design and construct resembles the real worldobject it seeks to represent. For example, one may run a helperapplication for a music file. After clicking on the icon for the helperapplication, the 3D GUI may prompt the user or automatically locate themusic file(s) on either their local computer, networked computer orworld wide webservice. The helper application may then draw theinterface for the music file(s) in the 3D space. The user can theninteract with the graphical representation of the music file (e.g., agraphical representation of a CD) to run and play the music file.

Regardless of where the file or application displayed in the 3D GUI islocated (e.g., within the same folder, subfolder, a different computer,within the network, a different network, across the internet, etc.), theuser has full access to the file through it's native program, or to thewebsite through the default browser. In this way, the 3D GUI allows theoutput of disparate computer programs to visually converge into oneexpandable, changeable 3D space. The 3D GUI may also prompt the user tosearch or scan folders and subfolders in the local computer, networkcomputers, webservers or the internet for any files of a given kind orcriteria and display their output as windows or customized 3D real worldobject icons en masse in the 3D space via the helper application. Thisway, the user can use this feature selectively to choose only one fileor automatically to choose all the files available to them.

In accordance with another aspect of the embodiments described herein,the end user is provided with the ability to selectively capturecomputing output and information (webpages, applications, documents,desktops or anything that can be visualized on a computer) and allowingit to visually converge within a 3D virtual space. In one embodiment ofthe invention, the 3D GUI offers the ability to drag-and-drop content inthe form of files, folders, applications, windows, documents or anythingelse expressed on the end user's desktop (or networked desktop) to a 3Dvirtual space by locating it on the desktop or within a window on thedesktop and dragging the icon(s) of the item (or the open item throughits window) to the 3D GUI window's virtual space to add it to. Drag anddrop describes a particular action you can make with the mouse. Click anobject, such as a folder, then hold down the mouse button as you dragthe object to a new location. One can drop the object by releasing themouse button. Drag and drop can be used to easily move or embed anobject or file into another.

Once the icons are dragged into the virtual space, the document,application, file or other can (i) open within a window in the virtualspace or (ii) can be represented by its icon within the virtual space.In one embodiment of the invention, if the drag-and-dropped item isrepresented by an icon within the virtual space, it can then bedouble-clicked to open it within the virtual space or outside of thevirtual space on the 2D desktop. For example, the icon of the InternetExplorer application within a virtual space can be dragged-and-droppedinto the 3D GUI and shown as an icon.

If the end user double clicks this icon, the interne explorer window canopen within the 3D virtual space. In one approach, if the end userdouble clicks this icon and holds the shift key at the same time, theInternet Explorer window will open in front of the 3D virtual space in a2D window as part of the desktop. Each time the end user completes onedrag-and-drop operation, all of the items in the single drag-and-dropare filed in their own 3D stack and an icon is plotted on the timelineto represent this.

In another embodiment of the invention, if an end user drags-and-drops afolder containing multiple items from the desktop, all of those itemsthat are in the folder are drawn individually (outside of the folder) intheir own 3D stack within the 3D virtual space. In addition, an iconrepresenting this visual event through a drag-and-drop action by the enduser to alter the 3D virtual space is preferably added to the timelinein accordance with its function. In this case, an icon of a folder isdrawn on the timeline representing this action (drag-and-drop of afolder into the virtual space) as it is indexed in the timeline

In accordance with one aspect of the present invention, the end user canre-order, move around or further organize items plotted within the 3Dvirtual space automatically through the script or program of a helperapplication (FIGS. 1A-1C—box 32) written for a specific purpose such assorting or manually by clicking on one item or multiple items as a group(ctrl-click) and moving them to their new location throughdrag-and-drop. Drag and drop describes a particular action you can makewith the mouse. Click an object, such as a picture or webpage, then holddown the mouse button as you drag the object to a new location. You dropthe object by releasing the mouse button.

For example in one embodiment, illustrated in FIG. 17A, one can see fiveitems (590, 592, 594, 596, and 598) in the virtual space 300, whereinthe five items are represented in the timeline 340 by icons 600, 602,604, 606, and 608, respectively. The end user has the ability to moveany item in the 3D virtual space to any other location within thevirtual space by translating it along the x, y axis simply by clickingit, holding down the mouse button, dragging the object to the newlocation and releasing the mouse button. For example, if the end userclicks the webpage 590, holds down the mouse button and drags thewebpage to the left (−x) and up (+y) within the 3D virtual space andreleases the mouse button, the webpage will occupy a new location (seeFIG. 17B). The end user can also move items closer to their viewpoint(appearing larger) or further away from them (appearing smaller) withinthe virtual space by translating them along the z axis. In order toaccomplish the translation of an item within the virtual space in the −z(further away) or +z (closer) direction, the end user holds down theshift key at the same time they initiate a drag-and-drop of said item.For example, with reference to FIGS. 17A-17C, if the end user clicks theimage 596, holds down the mouse button and shift key down at the sametime and drags the mouse backwards closer to their person, the picture596 will be translated forward in the +z direction, appearing larger,and will occupy a new location in the virtual space. Similarly, if theend user clicks a webpage, holds down the mouse button and shift keydown at the same time and drags the mouse away from their person in aforward direction, the webpage will be translated backward in the −zdirection, appearing smaller, and will occupy a new location in thevirtual space (not illustrated).

In accordance with one aspect of the present invention, the user is ableto access helper applications while immersed in a 3D interactive scene,by clicking on icons located on a task bar at the base of the screen. Byclicking on these animated 3D icons, the icons may duplicate themselvesor animate themselves into the 3D scene and provide the beginning offunctionality as 3D graphical objects for the tasks at hand for whichthe icon was initially clicked. For example, the icon on the task bar toinitiate this helper application may be of an open doorway. Once the 3Dopen doorway icon on the task bar is clicked, a picture of a viewpointor visual perspective of the new space may be animated from the task barinto the scene. If this picture were to be clicked from within the 3Dscene, it would act as a doorway that would hyperlink the user from thecurrent scene to another scene.

In accordance with one aspect of the present invention, the display of3D on a computer screen may involve a run-time execution model thatplays a previously programmed 3D space like a tape-recording in a windowor web browser as a one-way broadcast medium. The programming of this 3Dspace is accomplished by hand coding a program or using an editor whichprepares a file for the player. A helper application for geometry can beaccessed from the player (while an end user is using and immersed withina 3D space) through a graphical user interface of interactive icons thatfacilitate the display or input of 2D and 3D objects into the 3D scenebased on a user interface of geometrical objects (cube, spheres, cones)or real world objects (room, desk, building, stairs, CD rack). Thisaddition of geometry into the 3D scene on-the-fly helps give meaning toan otherwise endless connection of 3D spaces for which users can filltheir output with. The 3D GUI may further include a special Search andBrowse application to locate 3D geometry files. The user may use theSearch and Browse application, for example, to search for geometry filesin (or beyond) folders and subfolders located on the local computer,network computers, webservers or the world wide web.

In accordance with one aspect of the present invention, the display of3D on a computer screen may involve a run-time execution model thatplays a previously programmed 3D space like a tape-recording in a windowor web browser as a one-way broadcast medium. The programming of this 3Dspace is accomplished by hand coding a program or using an editor whichprepares a file for the player. A helper application for audio, soundand music can be accessed from the player (while an end user is usingand immersed within a 3D space) through a graphical user interface ofinteractive icons that provide pre-recorded audio, sound and music andmay also facilitate the recording of sound through a microphoneconnected to the computer and inserted into the 3D scene through an icon(e.g., of a speaker, etc.) on-the-fly as a way to label (or narrate) thescene and record thoughts next to objects in the scene to further givethem meaning. The user may further use the Search and Browseapplication, for example, to search for audio, sound, or music files in(or beyond) folders and subfolders on the local computer, networkcomputers, webservers or the world wide web.

In accordance with one aspect of the present invention, the display of3D on a computer screen may involve a run-time execution model thatplays a previously programmed 3D space like a tape-recording in a windowor web browser as a one-way broadcast medium. The programming of this 3Dspace is accomplished by hand coding a program or using an editor whichprepares a file for the player. The 3D GUI is unique in that it combinesauthoring capabilities to the end user within the interactive 3D virtualspace (FIG. 2—boxes 22, 24, 26). For example, a helper application forpictures, animations, movies and video can be accessed from the playerthrough a graphical user interface of interactive icons that help locatesuch pictures, animations, movies and video and insert such media intothe 3D scene on-the-fly at the end user's behest (see FIGS. 1A-1C—boxes74, 86 while a user is using and immersed within a 3D space).

In one embodiment, shown in FIG. 13A, the end user selects the Desktophelper application 451 in the database module 440. The selection of thedesktop tab 451 reveals files on the end user's local hard drive ornetwork in the window pane or menu 441 below. One can view files shownon the end user's local hard drive within this window pane 441. One canalso drag-and-drop (FIGS. 1A-1C—box 78) one or more of these files intothe GUI 3D virtual space 300 which results in the display of saidpicture, video and webpage incorporated into the 3D virtual space alongwith their 3D interactive icons represented in the timeline 340 of the3D virtual space 300.

In another embodiment, shown in FIG. 15, content output into the 3D GUIapplication's virtual space results from a search initiated with thehelper application 522 (e.g., Yahoo! Image Search). This helperapplication shows the search term van gogh, having been input into thishelper application's text input field 530. The resulting first fourimage outputs (552, 554, 556, 558), generated by helper application 522(see functional diagrams in FIGS. 4A and 4B), created the 3D output ofimages and information from Yahoo! webservice, as shown in the 3Dvirtual space. The beginnings of a map or timeline 340 are provided(namely, an icon 550 on the bottom margin of the virtual space) wherebythe 3D GUI is adapted to express the map as a timeline of storedsearches through 3D icons with their names (should the end usermouse-over them). Here, the timeline 340 is a map which represents thelinear progress of events within the virtual space. Clicking the 3Dicons (e.g., icon 550) would animate from the first viewpoint of thefirst item in each 3D stack to the last viewpoint of the first item inthe last 3D stack should the end user click each 3D icon in such asequence, originally indexed in the database module.

In essence the 3D icon 550 is a like hyperlink—namely, a graphic thatjumps (when clicked) to a new location or viewpoint in the currentvirtual space corresponding to the 3D icon. In the exemplary embodimentof FIG. 15, each new Yahoo! Image Search results in a new 3D stackcreated for that search result plotted in the 3D virtual space with itscorresponding 3D icon drawn on the timeline. The search for informationcould be images, video or any other content available from a webservice.The user may further use the Search and Browse application, for exampleto search for pictures, animations, movies and video files in (orbeyond) folders and subfolders on the local computer, network computers,webservers or the world wide web.

In accordance with another aspect of the present invention, the displayof 3D on a computer screen may involve a run-time execution model thatplays a previously programmed 3D space like a tape-recording in a windowor web browser as a one-way broadcast medium. The programming of this 3Dspace is accomplished by hand coding a program or using an editor whichprepares a file for the player. The 3D GUI is unique in that it combinesauthoring capabilities to the end user within the interactive 3D virtualspace. In one embodiment, illustrated in FIG. 14, the user can access apaintbrush feature by selecting the 3D helper application tab 460 in thedatabase module or compass 440, and then selecting the Add Paint Brushfeature in the menu 441. A user interface of interactive icons 542, 544appear in the virtual space 200, wherein the icons 542, 544 facilitatethe display or output of 2D and 3D free formed lines and drawings 546based on mouse events painted into the scene on-the-fly as a way toannotate or decorate objects in the scene or create drawings next toobjects to further give them meaning. More specifically, clicking onpaintbrush icon 542 causes a painting tool to appear in the virtualspace 300.

In this embodiment of the invention, the 3D GUI system redraws the 3Dvirtual space in a cyclical fashion to reflect the changes of contentand perspective within the 3D space based on the end user's input (seeFIGS. 1 and 2—box 22). The program scans the event handler for inputfrom one of the many helper applications (FIGS. 1A-1C—box 30). In thispresent exemplary embodiment, the helper application is the Add PaintBrush. The end user chooses the paintbrush icon 542, which changes theend user's cursor to the brush icon. The 3D GUI captures the mousemovements of the end user and draws or paints this movement (if themouse button is depressed) which in this example is (i) an arrow, (ii)an underline and (iii) the word important painted in red (546) on to thex and y plane of the end user's viewpoint within the virtual space's 3DCartesian space in the color red.

In accordance with another aspect of the present invention, there isprovided a helper application for text that can be accessed from theplayer (while a user is using and immersed within a 3D space) through agraphical user interface of interactive icons that facilitate thedisplay or output of 2D and 3D text into the scene on-the-fly as a wayto label objects in the scene and write down thoughts next to objects inthe scene to further give them meaning.

In one embodiment, the helper application is an Add Text Command (FIGS.1A-1C—box 72). For example, as shown in the embodiment of FIG. 14, whenthe end user chooses the add text command by clicking on the icon 544,the end user's cursor changes to the text I-beam icon. The 3D GUIcaptures the keyboard input of the end user and draws or paints thistext onto the x and y plane of the end user's viewpoint within thevirtual space's 3D Cartesian space.

In accordance with another aspect of the present invention, the displayof 3D on a computer screen may involve a run-time execution model thatplays a previously programmed 3D space like a tape-recording in a windowor web browser as a one-way broadcast medium. The programming of this 3Dspace is accomplished by hand coding a program or using an editor whichprepares a file for the player. Once an end user is viewing a 3D spaceor scene on their computer through a player on either their web browseror stand alone application, a Search application can be used to input asearch term or search terms.

The 3D GUI will then conduct a world wide web search using these searchterms in at least one search engine (Google, MSN, Yahoo!) by opening one2D HTML page drawn into a window or other object into the 3D scene foreach of the search engines to display the output of the HTML page foreach search. The 3D GUI may then parse the search results found in eachHTML page and identify each hyperlink that the search engine returned tothe end user as a search result. Once identified, the 3D GUI may thenopen one new window in the 3D space (behind the search results page) forat least one hyperlink found on the search results page that woulddisplay the webpage found at the URL for the Hyperlink (e.g., next setof search results, a particular search results, a particularadvertisement, etc.).

In one embodiment of the present invention, the 3D GUI may do this foreach hyperlink found on the search results page en masse for one or moresearch engines, and tile them in space. This plotting of search resultsinto a 3D scene is beneficial to consumers as it expands the visualcomputing space one has available for both web searches and internetbrowsing.

In another embodiment of the present invention, the 3D GUI makes use ofdormant computing time while the end user is connected to a network. Forexample, most users will conduct a search, scan the page and click onelink. Once they realize that the link they clicked was insufficient,they will click the back button on their browser, scan the page again,and click another link. The time the consumer is scanning the page,clicking one link, reading the page at the new link and returning backto the original search results page is dormant computing time where thecomputer is connected to the world wide web and able to download all (orportions of) the hyperlink pages found on the original search resultspage, continuously, and draw these results into the 3D scene. The 3D GUIallows users to systematically visit a viewpoint that shows each searchresults page one at a time for easy viewing. The graphical userinterface will have options for saving the current search results into afile and post the name of the search, name of the hyperlink, URL,date/time and/or source onto the database module.

Using this same seemingly unlimited 3D visual space, the 3D GUI mayallow the user to automatically parse any open or active webpage to (i)determine which HTML pages are links to it, (ii) determine which HTMLpages it links to, and (iii) plot these pages as groups or clusters.There may further be a visual grouping or cluster of pages drawn in a 3Dspace that link to any given active webpage. In this way, the user candetermine in one glance at a given viewpoint of a clustering of webpagesin the 3D space which pages to click that either link to their activepage or from their active page. In addition, the user may be able tovisually expand this network of webpages by choosing any active webpagein any cluster drawn in the 3D space and creating a new cluster ofwebpages that this active page links to or by creating a new cluster ofwebpages that is linked to this active page. In this way, the 3D GUIcreates a visual gestalt of the world wide web at the behest of theuser.

In accordance with another aspect of the present invention, the displayof 3D on a computer screen may involve a run-time execution model thatplays a previously programmed 3D space like a tape-recording oranimation in a window or web browser as a one-way broadcast medium. Theprogramming of this 3D space is accomplished by hand coding a program orusing an editor which prepares a file for the player. Once an end useris viewing a 3D space or scene on their computer through a interactiveplayer on either their web browser or stand alone application, the 3DGUI may deliver textual, audio, video, multimedia, web advertisementsand/or 3D advertisements whose presentation is automaticallyincorporated into the programming that is responsible for the currentview of 3D. This way, based on user inputs that help define the subjector context for which they are either searching or computing, the 3D GUImay deliver one or a combination of advertisements (e.g., advertisementspreviously mentioned into the 3D space, etc.).

The 3D GUI described herein is an improvement over existing players thatplay 3D animation in that 3D GUI described herein anticipates an enduser's input to navigate and add content to the 3D virtual spaces(through helper applications), on-the-fly, that it allows an end user tocreate; merging the editor concept and player concept of 3D into onehybrid editor and player. In doing so, the GUI system redraws the 3Dvirtual space in a cyclical fashion to reflect the changes of contentand perspective within the 3D space based on the end user's input (FIGS.1 and 2—box 22). The program scans the event handler for input from oneof the many helper applications (FIGS. 1A-1C—box 30).

The helper applications can initiate the output of content into the 3Dvirtual space such as but not limited to (i) loading a custom script andits icon to alter the application logic of the 3D virtual space throughprogrammatic access to stream in new information and take on a new theme(a.k.a. helper application such as eBay Search, (ii) visit a viewpointcommand to change the visual perspective of the end user in the currentvirtual space or another virtual space that is hyperlinked to, (iii)record the current viewpoint in the virtual space, (iv) add the outputof the desktop the virtual space, (v) add a web browser to the virtualspace to view a document, webpage or files and folders on the desktopthrough a window in the 3D virtual space, (vi) add text command tovirtual space, (vii) add a picture to the 3D virtual space, (viii) addmusic to the 3D virtual space (FIG. 22—Item A), (iv) add video to the 3Dvirtual space, (x) record sound and at add it to the 3D virtual space,(xi) add a map to the 3D virtual space, (xii) add 3D through a VRML, X3Dor other file format to the 3D virtual space, and/or (xiii) addadvertising to the 3D virtual space in the form of text, pictures, videoor other multimedia (e.g., the embodiment of FIG. 19, described infurther detail below). The addition of such content to a 3D virtualspace within the 3D GUI system is diagrammed in FIGS. 1A-1C—boxes 32,42, 50, 58, 60, 62, 72, 74, 84, 86, 88, 90, 92, 94, 104.

Once the 3D GUI scene recalculates the geometry in the scene (FIGS. 1and 2—box 24) based on the addition of output, information and contentto the 3D virtual space it then redraws the scene (FIGS. 1 and 2—box26), to reflect the addition of this new content. The 3D GUI system willdetermine the subject or meaning of the output, information and contentwithin the 3D virtual space by indexing all the words associated with(i) the names of the files, HTML pages, viewpoints, hyperlinks and anyother available information associated with the 3D virtual space contentin the scene, (ii) actual data within the files, HTML pages, viewpoints,hyperlinks and other available information within the 3D virtual spaceor words entered into the input field of helper application (FIGS.1A-1C—box 106), (iii) sort the indexed words by frequency to determinesubject of 3D virtual space by identifying most frequently appearingwords or utilize other method to determine meaning, (iv) send subject of3D virtual space to advertising server through internet request (FIGS.1A-1C—box 110), (v) return appropriate text, picture(s), video, sound,music, hyperlinks or other advertisement content and respective softwarecode from advertising server, webservice or other location, (vi) drawtext, picture(s), video(s), sound(s), music, hyperlinks or other adcontent from advertising server or other location into 3D virtual spaceusing program code (FIGS. 1A-1C—box 114) and (vii) redraw the scene(FIGS. 1 and 2—box 26). Those skilled in the art may identify thisprocess of making an advertising request through the internet as manyweb sites utilize this technology for delivering advertising through theinternet.

In one embodiment, shown in FIG. 19, when the end-user inputs the searchterm news into the helper application 640 (e.g., Yahoo! Search), thehelper application 640 returns a webpage 642 (e.g., URLhttp:www.ncn.com). The 3D GUI system can further utilizes the input term(in this case news) (see FIGS. 1A-1C—box 104) and returns a group ofadvertising text 644. The advertising text 644 can be incorporated intothe 3D virtual space 300 by presenting the text along with its: (a)hyperlink and associated title (News on eBay); (b) description (Findnews items at low prices. With over 5 million items for sale every day,you'll . . . ); and (c) url www.ebay.com, thereby resulting in a moredetailed advertisement 646. Currently, text advertisements are prevalenton the world wide web and therefore have been incorporated in thisembodiment of the invention. However, it will be understood that otherforms of advertising, such as, for example, picture(s), video(s),sound(s), music, hyperlinks, and/or multimedia, and/or combinationsthereof, can be incorporated into the 3D virtual space 300.

The present invention provides many advantages over the prior art,including, but not limited to, increased space on the user's desktop,eliminating the need to constantly open and close programs or hide andreveal them each time the user needs them, utilizing dormant computingtime to download and/or display information, reducing mouse-clicks andoffering a more natural alternative to accessing programs and documentsthan the traditional folders-within-folders storage system. For example,in the exemplary embodiment of FIG. 12, forty webpages are output intothe 3D virtual space 300 organized into their respective 3D stacks 470,472, 474, 476, and 478, wherein the stacks are represented in thetimeline 340 as icons 480, 482, 484, 486, and 488, respectively. Asexplained above, the present invention also improves the web browser anddesktop experience by offering a novel 3D graphical user interface.

In accordance with yet another aspect of the embodiments describedherein, there is provided a memory management method whereby computermemory is allocated to the 3D GUI system at the program's request andfreed up for reuse when it is no longer needed by unloading memory. Morespecifically, there is provided a method of using virtual memory wherebya portion of the hard disk is used to swap out this data wheninsufficient RAM exists to hold all such data. In one embodiment, an enduser plots the output of information into the 3D GUI within 3D stackswhereby each new visual event additionally marked with an icon on thetimeline. Additional output of information into the 3D GUI virtual spaceresults in the creation of new 3D stacks along with new icons drawn insuccession on the timeline creating what is called linear path of theend user's actions through a virtual space as expressed with through atimeline and 3D stack. If this process continues indefinitely, thememory or electronic holding place for this data that the 3D GUI systemcan access quickly will be used up. In order to create an endingcomputer experience whereby the end user need not reach such a memorylimit, provided herein is a system whereby once a memory limit isreached, the 3D GUI system marks a point in the program representing theposition of the end user on this linear path and unloads the memory bysaving it to virtual memory. This freeing up of memory will allow theend user to continually output new information into the virtual space.This process of unloading memory to virtual memory and continuallyoutputting new information into the virtual space can go onindefinitely. Should the end user backtrack or retrace one's course overa portion of the linear path (or track) that has already been completed,the 3D GUI system will reload to memory the information stored invirtual memory once the end user revisits any mark on the linear path.

The present invention can be used in a number of applications,including, but not limited to, search engines, desktops, visual musicdownload services, shopping malls, collaborative space for shareddocuments, collaborative space for video conferencing, tool forpublishing webpages, virtual networked computer system, interface forcable TV sets or multimedia PCs, computing interface for large flatpanel displays, forum for educational training and visualization, ande-mail visualization programs, just to name a few. Even though thepresent invention is described here within the embodiment of anoperating system that utilizes a desktop personal computer with amonitor, those skilled in the art will be able to adapt the system towork on other electronic devices, such as, for example, cell phones,pdas, handheld mobile devices, flat panel displays, or the like, etc.,without giving up the spirit of the invention.

Having thus described a preferred embodiment of a method and system forproviding an improved three-dimensional graphical user interface, itshould be apparent to those skilled in the art that certain advantagesof the within system have been achieved. It should also be appreciatedthat various modifications, adaptations, and alternative embodimentsthereof may be made within the scope and spirit of the presentinvention. For example, the improved 3D GUI has been presented in thecontext of a windows operating system, but it should be apparent thatmany of the inventive concepts described above would be equallyapplicable for other operating systems and devices.

What is claimed is:
 1. A method for providing a three-dimensional (3D)graphical user interface, comprising: receiving at least first andsecond inputs from an end user; receiving first and second webpages fromat least one server in response to said first and second inputs, whereinthe first and second inputs are website addresses corresponding to saidfirst and second webpages, respectively; displaying at least a portionof the first webpage on a first object within a 3D space, and at least aportion of the second webpage on a second object within the 3D space,comprising; rendering the first and second webpages; capturing first andsecond images of the at least a portion of the first webpage and the atleast a portion of the second webpage, respectively; and texturing thefirst image on the first object and the second image on the secondobject, the first object being displayed in a foreground of the 3D spaceand the second object being displayed in a background of the 3D space;and displaying additional information, comprising: receiving aninteraction by the end user on the first image; replacing the first andsecond objects within the 3D space with a window within atwo-dimensional (2D) space in response to receiving the interaction,wherein the window includes the rendered first webpage; receiving aninteraction by the end user on a link provided in the rendered firstwebpage, the link corresponding to the additional information; renderingthe additional information; and displaying the rendered additionalinformation in said window within the 2D space.
 2. The method of claim1, further comprising: capturing a third image of at least a portion ofthe rendered additional information; texturing the third image on thefirst object, the third image thereby replacing the first image on thefirst object; and replacing the window within the 2D space with at leastthe first and second objects within the 3D space, wherein the firstobject is displayed in the foreground of the 3D space and the secondobject is displayed in the background of the 3D space.
 3. The method ofclaim 2, further comprising: receiving a toggle interaction by the enduser; and replacing the window within the 2D space with at least thefirst and second objects within the 3D space in response to the toggleinteraction.
 4. The method of claim 2, further comprising: receiving anavigation interaction by the end user; and moving said second objectfrom the background of the 3D space to the foreground of the 3D space inresponse to the navigation interaction.
 5. The method of claim 1,further comprising: receiving a toggle interaction by the end user; andreplacing the window within the 2D space with at least the first andsecond objects within the 3D space in response to the toggleinteraction.
 6. The method of claim 1, further comprising: receiving atleast a third input from the end user; receiving a third webpage fromthe at least one server in response to the third input; and displayingat least a portion of the third webpage on a third object within the 3Dspace, comprising: rendering the third webpage; capturing a third imageof the at least a portion of the third webpage; and texturing the thirdimage on the third object, the third object being displayed in a furtherbackground of the 3D space, behind the second object.
 7. The method ofclaim 1, wherein the step of receiving the first and second webpagesfrom the at least one server in response to said first and second inputsfurther comprises receiving the first webpage from a first server inresponse to said first input and receiving the second webpage from asecond server in response to said second input.
 8. A system forproviding a three-dimensional (3D) graphical user interface, comprising:a display screen; an input device for receiving at least one input froman end user a processor module operatively coupled to the display screenand the user input device; and a memory module operatively coupled tothe processor module, the memory module comprising executable code forthe processor module to: receive at least first and second inputs froman end user; receive first and second webpages from at least one sourcein response to said first and second inputs, wherein the first andsecond inputs are website address corresponding to said first and secondwebpages, respectively; display at least a portion of the first webpageon a first object within a 3D space on the display screen, and at leasta portion of the second webpage on a second object within the 3D spaceon the display screen, comprising; rendering the first and secondwebpages; capturing first and second images of the at least a portion ofthe first webpage and the at least a portion of the second webpage,respectively; and texturing the first image on the first object and thesecond image on the second object, the first object being displayed in aforeground of the 3D space and the second object being displayed in abackground of the 3D space; and display additional information,comprising: receiving an interaction by the end user on the first image;replacing the first and second objects within the 3D space with a windowwithin a two-dimensional (2D) space on the display screen in response toreceiving the interaction, wherein the window includes the renderedfirst webpage; receiving an interaction by the end user on a linkprovided in the rendered first webpage, the link corresponding to theadditional information; rendering the additional information; anddisplaying the rendered additional information on the display screen insaid window within the 2D space on the display screen.
 9. The system ofclaim 8, wherein said executable code is further configured to: capturea third image of at least a portion of the rendered additionalinformation; texture the third image on the first object, the thirdimage thereby replacing the first image on the first object; and replacethe window within the 2D space with at least the first and secondobjects within the 3D space, wherein the first object is displayed inthe foreground of the 3D space and the second object is displayed in thebackground of the 3D space.
 10. The system of claim 9, wherein saidexecutable code is further configured to: receive a toggle interactionby the end user; and replace the window within the 2D space with atleast the first and second objects within the 3D space in response tothe toggle interaction.
 11. The system of claim 9, wherein saidexecutable code is further configured to: receive a navigationinteraction by the end user; and move said second object from thebackground of the 3D space to the foreground of the 3D space in responseto the navigation interaction.
 12. The system of claim 8, wherein saidexecutable code is further configured to: receive a toggle interactionby the end user; and replace the window within the 2D space with atleast the first and second objects within the 3D space in response tothe toggle interaction.
 13. The system of claim 8, wherein saidexecutable code is further configured to: receive at least a third inputfrom the end user; receive a third webpage from the at least one serverin response to the third input; and display at least a portion of thethird webpage on a third object within the 3D space, comprising:rendering the third webpage; capturing a third image of the at least aportion of the third webpage; and texturing the third image on the thirdobject, the third object being displayed in a further background of the3D space, behind the second object.
 14. A method for providing athree-dimensional (3D) graphical user interface, comprising: receivingat least first and second website addresses from an end user; using saidfirst and second website addresses to retrieve first and second webpagesfrom at least one source in response to said first and second inputs;displaying at least a portion of the first webpage within a 3D space,and at least a portion of the second webpage within the 3D space,comprising; generating first and second images of the at least a portionof the first webpage and the at least a portion of the second webpage,respectively; and displaying the first image and the second image in the3D space, the first image being displayed in a foreground of the 3Dspace and the second image being displayed in a background of the 3Dspace; and displaying additional information to said end user,comprising: receiving an interaction from the end user with the firstimage; replacing the first and second images within the 3D space with awindow within a two-dimensional (2D) space in response to receiving theinteraction, wherein the window includes the first webpage; receiving aninteraction by the end user on a link provided in the first webpage, thelink corresponding to the additional information; and displaying theadditional information to the user.
 15. The method of claim 14, whereinthe additional information is displayed in the window, thereby replacingthe first webpage in the window.
 16. The method of claim 14, furthercomprising: generating a third image of at least a portion of theadditional information; and replacing the window with at least thesecond and third images within the 3D space, wherein the third imagereplaces the first image in the foreground of the 3D space, and thesecond image remains in the background of the 3D space.
 17. The methodof claim 16, further comprising: receiving a toggle interaction by theend user; and replacing the window with at least the second and thirdimages within the 3D space in response to the toggle interaction. 18.The method of claim 17, further comprising; receiving a navigationinteraction by the end user; and moving said second image from thebackground of the 3D space to the foreground of the 3D space in responseto the navigation interaction.